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Abstract 

Inefficiencies  in  the  healthcare  system  are  a  growing  concern.  Long  wait-times 
are  a  concern  at  military  clinics  because  they  take  servicemembers  away  from  performing 
their  duties.  Managing  wait-times  is  particularly  challenging  due  to  frequent  relocations 
of  servicemembers  and  variable  patient  demands  that  are  less  likely  to  be  experienced  by 
civilian  clinics.  Military  clinics  must  be  capable  of  meeting  increasing  demand  when 
servicemembers  require  a  Deployment  Health  Assessment;  they  also  need  to  be  capable 
of  handling  an  instantaneous  surge  of  walk-ins  when  a  medical  incident  occurs  in  the 
local  area.  They  must  be  able  to  meet  these  demands  in  a  fiscally  austere  environment. 

Existing  research  primarily  focuses  on  stand-alone  clinics,  whereas  this  research 
takes  a  novel  approach  of  examining  a  system  of  clinics,  in  which  some  resources  are 
shared.  This  research  evaluates  the  impacts  of  variable  staffing  levels  on  total  wait-time 
for  the  system  of  clinics  at  baseline  demand  and  when  demand  increases,  using  discrete- 
event  simulation,  sensitivity  analysis,  and  cost-benefit  analysis.  This  research  finds 
misallocated  resources;  the  wait-time  of  alternative  systems  are  sensitive  to  deployment 
and  medical  incident  demands;  and  hiring  an  optometrist  while  removing  an  occupational 
medicine  doctor  provides  the  highest  savings  in  baseline,  deployment,  and  medical 
incident  demand  environments. 
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A  PROCESS  IMPROVEMENT  STUDY  ON  A  MILITARY  SYSTEM  OF  CLINICS 
TO  MANAGE  PATIENT  DEMAND  AND  RESOURCE  UTILIZATION  USING 
DISCRETE-EVENT  SIMULATION,  SENSITIVITY  ANALYSIS,  AND 
COST-BENEFIT  ANALYSIS 

I.  Introduction 

Chapter  Overview 

This  chapter  provides  an  overview  of  the  thesis  topic.  It  provides  a  background  of 
the  healthcare  system  as  well  as  states  the  issues  within  the  system  of  clinics.  It  also 
presents  the  research  question,  defines  the  focus  of  the  research  to  a  system  of  clinics  at 
Wright-Patterson  Air  Force  Base,  and  establishes  three  investigative  questions.  It  then 
declares  the  method  to  answer  the  investigative  questions  using  discrete-event  simulation, 
sensitivity  analysis,  and  cost-benefit  analysis,  addresses  the  limitations  of  the  study,  and 
illustrates  the  assumptions  made.  Finally,  it  comments  on  the  potential  implications  of 
the  study  to  the  military  healthcare  system  and  describes  the  subsequent  chapters  of  this 
thesis. 


Background 

Inefficiencies  in  the  healthcare  system  are  receiving  nationwide  public  attention 
through  the  media.  The  Associated  Press  (2012)  reports  that  healthcare  systems  waste 
billions  of  dollars.  Even  President  Barack  Obama  (2014)  has  concerns  over  the 
inefficiencies  in  the  healthcare  system.  With  variable  demands  of  healthcare  services, 
inefficiencies  in  the  system  have  to  be  eliminated  in  order  to  sustain  healthcare  services 
over  the  next  decade.  For  the  military,  long  wait-times  are  of  particular  concern  for 
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active  duty  personnel  since  the  long  wait-times  take  these  personnel  away  from  their 
official  duties. 

Due  to  the  uncertainty  of  future  crises,  military  service  members  can  expect  short- 
notice  deployments.  Ideally,  military  readiness  needs  to  be  at  100%  at  all  times.  If  a 
crisis  occurs  requiring  the  military  to  deploy,  then  military  healthcare  clinics  must  be 
capable  of  medically  clearing  personnel  for  deployment  in  a  timely  fashion.  Not  only  do 
military  clinics  need  to  meet  the  demand  of  deployments,  they  also  need  to  be  capable  of 
handling  an  instantaneous  surge  of  walk-ins  when  a  mild  medical  incident  occurs  on  the 
military  installation. 

Unfortunately,  the  military  currently  faces  a  fiscally  austere  environment.  The 
military’s  budget  is  being  reduced  by  billions  of  dollars  over  the  next  two  years  and 
personnel  end  strength  reduced  by  tens  of  thousands  (Simeone,  2014).  As  defense 
budgets  become  constricted,  senior  leaders  of  the  military  healthcare  system  need  to  find 
ways  to  improve  current  healthcare  processes  in  order  to  maintain  its  level  of 
performance  as  patient  demand  changes.  The  71 1th  Human  Performance  Wing  from  the 
Air  Force  Research  Laboratory  is  sponsoring  this  research  to  acquire  solutions  in 
improving  the  military  healthcare  system  at  Wright-Patterson  Air  Force  Base. 


Research  Question 

How  can  the  total  wait-time  patients  experience  in  the  military  system  of  clinics 
be  cost-effectively  reduced  during  baseline  demand  and  when  patient  demand  increases 
as  the  clinics  within  the  system  of  clinics  compete  for  scarce  resources?  A  system  of 
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clinics  is  defined  as  a  group  of  clinics  co-located  in  one  building  where  certain  medical 


resources  are  shared.  A  military  environment  is  defined  as  situations  where  active  duty 
military  personnel  are  deployed  as  a  normal  aspect  of  daily  working  conditions. 


Investigative  Questions 

There  are  three  investigative  questions  that  must  be  addressed  in  order  to  answer 
the  research  question: 

1. )  How  can  staffing  levels  be  adjusted  to  improve  the  patient’s  overall  wait-time  in  the 
system  of  clinics?  The  patient’s  overall  wait-time  in  the  system  of  clinics  is  the 
dependent  variable.  Staffing  levels  are  the  independent  variables  that  could  potentially 
affect  the  overall  wait-time  of  the  patient. 

Hypothesis:  One  or  more  staffing  levels  will  have  statistically  lower  wait-times 
than  the  baseline  staffing  level  of  the  system. 

2. )  Which  staffing  level  solution  is  the  most  robust  as  patient  demand  increases? 
Robustness  is  defined  as  the  ability  to  maintain  the  level  of  performance  of  the  system  of 
clinics’  as  patient  demand  changes. 

Hypothesis:  One  or  more  staffing  levels  will  have  statistically  lower  wait-times 
than  other  staffing  levels  when  exposed  to  a  surge  in  patient  demand. 

3. )  Which  system  improvement  solution  has  the  lowest  cost  to  implement?  Cost  to 
implement  includes  two  variables.  First,  monthly  salary  being  paid  to  the  staff  member, 
based  on  type,  is  accounted  for.  Second,  the  cost  equivalent  of  wait-time  reduced  or 
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wait-time  increased  is  calculated.  A  reduction  in  wait-time  is  a  desired  effect  of  an 


alternative  system  implementation  so  such  an  implementation  has  a  positive  dollar 
equivalent.  Conversely,  an  increase  in  wait-time  is  considered  an  adverse  effect  of  an 
alternative  system  implementation  so  this  implementation  has  a  negative  dollar 
equivalent. 

Hypothesis:  One  or  more  staffing  levels  will  have  a  statistically  lower  cost  to 
implement  than  other  staffing  level  alternatives. 


Research  Focus 

The  focus  of  this  study  is  to  model  the  behavior  of  a  current  process  in  a  system 
of  clinics  located  at  Wright-Patterson  Air  Force  Base  (WPAFB)  and  to  identify 
alternative  processes  that  will  improve  the  overall  performance  of  the  system  of  clinics. 
The  study  will  focus  primarily  on  patient  processing  through  the  system  of  clinics.  Five 
clinics  are  investigated  utilizing  shared  resources  all  co-located  in  a  single  building. 
These  five  clinics  are  the  Flight  Medicine  Clinic,  the  Occupational  Medicine  Clinic,  the 
Hearing  Conservation  Clinic,  the  Audiology  Clinic,  and  the  Optometry  Clinic. 


Methodology  Overview 

The  overarching  research  goal  is  to  identify  feasible  solutions  to  the  research  question. 
To  accomplish  this,  this  research  uses  a  five-step  process.  The  first  step  is  to  assess  the 
operational  behavior  of  the  system  of  clinics.  To  achieve  this  objective,  a  data  collection 
effort  must  be  conducted.  The  span  of  this  effort  spanned  one  month  in  August  2014. 
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Data  is  collected  for  each  of  the  studied  clinics  by  having  the  medical  staff  collect  time 


data  of  the  patient  as  they  process  through  the  different  stations  of  various  clinics.  The 
second  step  is  to  build  a  baseline  discrete-event  simulation  model.  The  model  is  based  on 
the  data  collected.  It  is  verified  with  the  medical  staff  to  acknowledge  that  the  model  is 
an  adequate  representation  of  the  system  of  clinics.  It  is  also  validated  using  data 
collected  from  the  system  of  clinics.  The  third  step  is  to  perform  experiments  using  the 
model.  The  experiments  aid  in  identifying  where  the  efforts  should  be  focused  in  order  to 
affect  the  desired  outcome.  The  experiments  evaluate  alternative  systems  that  can  affect 
the  patient  wait-time.  The  fourth  step  is  to  perform  a  sensitivity  analysis  on  the  feasible 
solutions.  Sensitivity  analysis  is  used  to  evaluate  the  robustness  of  a  feasible  solution 
when  subjected  to  changing  patient  demand.  The  fifth  step  is  to  perform  a  cost-benefit 
analysis  on  the  feasible  solutions.  Cost-benefit  analysis  is  used  to  evaluate  the  costs  to 
implement  on  alternative  systems  when  subjected  to  changing  patient  demand.  After  the 
five  steps  are  met,  the  results  stemming  from  these  tasks  are  used  to  formulate  a  solution 
to  the  overall  research  question  in  the  form  of  a  recommendation  which  is  presented  in 
Chapter  V  of  this  thesis. 


Methodology  Details 

To  address  each  investigative  question,  this  study  utilizes  discrete-event 
simulation  (DES),  sensitivity  analysis,  and  cost-benefit  analysis.  First,  DES  is  used  to 
answer  the  first  investigative  question.  A  conceptual  model  of  the  system  of  clinics  is 
created  in  order  to  build  a  DES.  A  baseline  model  is  then  created  in  ARENA  14.0  using 


5 


data  collected  by  the  staff  of  the  system  of  clinics.  It  is  then  verified  and  validated 
against  the  actual  system.  The  baseline  model  is  revised  until  there  is  statistical  evidence 
that  the  model  is  a  close  representation  of  the  real  system.  Once  the  baseline  model  is 
statistically  similar  to  the  real  system,  alternative  scenarios  of  the  system  are  simulated  in 
order  to  minimize  the  patient  wait-time. 

A  sensitivity  analysis  is  conducted  next  to  evaluate  the  robustness  of  the 
solutions.  The  analysis  explores  the  effect  of  patient  demand  changes  in  terms  of  a  200% 
demand  increase  due  to  increased  military  personnel  deployments  (deployment  demand). 
A  200%  increase  in  demand  equates  to  54  additional  patients  being  seen  throughout  the 
day.  A  second  sensitivity  analysis  is  conducted  in  terms  of  a  surge  in  patient  demand 
from  a  mild  medical  incident  occurring  on  the  military  installation  (medical  incident 
demand).  The  same  level  of  increase,  200%  or  54  additional  patients,  as  “walk-ins”  to 
the  clinic  for  the  first  three  hours  of  operation  is  studied  for  this  analysis.  Finally,  a  cost- 
benefit  analysis  is  conducted  to  evaluate  the  cost  and  benefit  trade-offs  of  implementing 
the  alternative  solutions.  The  analysis  looks  at  three  different  environments.  The  first 
environment  looks  at  the  system  of  clinics  when  demand  is  at  baseline  level.  The  second 
environment  looks  at  the  system  of  clinics  when  demand  increases  due  to  an  increase  in 
military  deployments.  The  third  environment  looks  at  the  system  of  clinics  when  there  is 
a  medical  incident  demand  for  the  first  three  hours  of  operation. 
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Limitations  and  Assumptions 

Due  to  resource  constraints,  limitations  are  imposed  upon  this  study.  One 
limitation  is  the  shortened  data  collection  period.  Time  data  of  the  patient  arriving  to  the 
system  of  clinics,  as  well  as  the  service  rate  data  of  each  of  the  stations,  are  not  readily 
available.  Because  data  are  not  currently  available,  a  data  collection  process  is  vital  to 
answer  the  research  question.  The  data  collection  period  is  limited  to  one  month,  August 
2014,  because  the  time  needed  to  process  more  data  is  unavailable  for  this  research.  If 
the  data  collection  period  is  over  the  entire  year,  then  all  the  months  of  the  year  can  be 
characterized  leading  to  a  more  complete  study.  Since  only  a  month’s  worth  of  data  is 
collected,  it  is  assumed  that  the  other  months  have  the  same  characteristics  as  the  month 
of  August.  Another  limitation  to  the  study  is  the  sample  of  the  study.  Due  to  limited 
funding,  the  researchers  were  not  authorized  a  Temporary  Duty  (TDY)  assignment  to 
travel  to  other  clinics  at  other  military  installations.  To  overcome  this  limitation,  this 
study  focuses  on  a  system  of  clinics  located  at  WPAFB;  system  of  clinics  at  other 
military  installations  may  behave  differently,  but  since  it  is  currently  not  feasible  to 
characterize  their  behavior,  it  is  assumed  that  they  have  the  same  characteristics  as  the 
system  of  clinics  at  WPAFB. 

With  the  inherent  complexity  of  a  system  of  clinics,  this  study  makes  several 
assumptions.  These  assumptions  are  necessary  in  order  to  create  a  simplified  model  that 
can  be  analyzed  towards  the  understanding  of  the  general  behavior  of  the  system  of 
clinics.  First,  the  analysis  assumes  that  only  patients  with  either  an  appointment  or  an 
acute  medical  condition  that  is  urgent,  but  not  serious  enough  to  go  to  the  emergency 
room  (ER)  at  a  hospital,  will  enter  the  clinic.  Another  assumption  is  that  the  patient  will 
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not  pre-maturely  leave  upon  entering  the  clinic.  This  is  known  as  balking.  These 
resource  constraints  are  the  reason  why  limitations  are  imposed  upon  this  research. 


Implications 

The  results  of  this  study  will  aid  the  WPAFB  system  of  clinics  to  implement 
economical  alternatives,  identify  bottlenecks  in  the  system,  reduce  patient  wait-time  and 
increase  the  utilization  of  its  medical  personnel.  If  this  study  is  replicated  throughout 
other  clinics  on  military  installations  across  the  United  States,  then  it  may  ultimately 
improve  the  military  healthcare  system.  It  will  assist  the  United  States  government  to 
utilize  taxpayer’  funding  more  effectively. 


Preview 

This  chapter  has  described  the  research  topic  and  provided:  pertinent  information 
on  the  current  problem,  the  objective  of  this  study,  the  approach  to  produce  a  solution  and 
the  potential  impacts  of  the  results  to  this  study.  The  following  chapter  of  this  thesis  will 
follow  a  traditional  format  for  Chapter  II,  Literature  Review.  The  remaining  chapters  of 
this  thesis  follow  a  scholarly  format  with  one  conference  paper  article  (Chapter  III)  and 
one  peer-reviewed  journal  article  (Chapter  IV)  followed  by  Chapter  V,  Conclusions  and 
Recommendations  Chapter. 
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II.  Literature  Review 


Chapter  Overview 

The  purpose  of  this  chapter  is  to  review  the  literature  that  fosters  an  understanding 
of  the  topics  discussed  in  subsequent  thesis  chapters.  This  chapter  explains  a  generalized 
overview  of  the  healthcare  industry,  simulation  in  healthcare,  sensitivity  analysis  in 
healthcare,  cost-benefit  analysis  in  healthcare,  and  the  research  gap  that  this  thesis 
addresses.  In  summary,  this  chapter  establishes  the  intellectual  foundation  of  the  subject 
areas  necessary  to  follow  the  discussion  throughout  the  thesis  chapters. 


The  Healthcare  Industry  Overview 

The  healthcare  system  is  one  of  the  most  important  systems  in  modern  societies. 
Without  the  healthcare  system  in  place,  members  of  society  would  find  it  a  challenge  to 
maintain  their  health  when  they  have  the  misfortune  to  experience  sickness  or  disease. 
The  expansion  of  the  healthcare  system  has  become  ubiquitous  in  American  society;  it  is 
“one  of  the  most  complex  business  models  in  American  industry  given  the  uniqueness  of 
the  marketplace  in  which  it  operates”  (Kudyba  &  Temple,  2010).  More  and  more  people 
are  relying  on  the  healthcare  system  to  help  them  find  relief  for  their  bodily  ailments,  and 
healthcare  providers  are  increasingly  being  forced  to  carry  the  financial  responsibility  of 
these  people  as  reimbursements  for  healthcare  services  rendered  are  dwindling  (Kudyba 
&  Temple,  2010).  As  a  result,  the  healthcare  industry  is  not  profiting  due  to  its  influx  of 
patients.  There  are  more  demands  for  healthcare  services  than  there  are  solutions  in  place 
to  meet  those  demands. 
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Simulation  in  Healthcare 


Simulation  is  widely  used  in  the  field  of  healthcare  to  discover  potential  solutions 
to  the  issue  of  system  inefficiencies  (Giachetti,  2008;  Kim,  et  al.,  2013;  Cote,  1999; 
Huschka,  Narr,  Denton,  &  Thompson,  2008;  Connelly  &  Bair,  2004;  Swisher,  Jacobson, 
Jun,  &  Balci,  2001;  Jacobson,  Hall,  &  Swisher,  2006).  For  example,  Giachetti  (2008) 
looks  at  using  simulation  to  combat  long  wait-times  patients  face  for  scheduled 
appointments.  Long  wait-times  due  to  overbooking  cause  patient  dissatisfaction.  Patient 
dissatisfaction  increases  patient  no-shows  the  next  time  an  appointment  is  booked  and,  in 
turn,  cause  a  reduced  output  of  clinical  care;  patient  no-shows  are  missed  opportunities 
for  other  patients  to  see  the  doctor.  Giachetti  found  that  removing  multiple  appointment 
types  can  reduce  patient  wait-time.  Giachetti  further  suggests  that  minimizing  patient 
wait-time  would  increase  patient  satisfaction  and  decrease  no-shows. 

DES  is  a  useful  tool  in  improving  and  establishing  individual  clinics.  Kim,  et  al. 
(2013)  found  that  adding  an  additional  psychiatrist  and  extending  daily  hours  of  an 
operation  by  two  hours  can  effectively  reduce  the  service  time  by  14.6  minutes,  on 
average,  in  order  to  improve  access  to  mental  health  services  at  a  mental  health  clinic. 
Cote  (1999)  used  DES  to  determine  the  capacity  of  examination  rooms  as  healthcare 
demands  increase,  finding  that  it  is  the  physician,  not  the  number  of  examination  rooms 
that  influences  the  quality  of  care.  Despite  the  physician  influencing  the  quality  of  care, 
if  physicians  are  consistently  over  utilized,  then  the  quality  of  care  diminishes.  This  is 
due  to  reduced  time  spent  by  the  physician  with  each  patient  in  order  to  meet  patient 
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demand  (Cote,  1999).  When  physicians  are  being  rushed,  they  also  increase  the  risk  of 
making  errors  which  result  in  patient  needs  not  being  met.  Physicians  typically  belong  to 
independent  clinics  where  they  do  not  share  resources  with  other  clinics.  Huschka,  et  al. 
(2008)  used  DES  to  establish  an  Outpatient  Procedure  Center  (OPC).  An  increase  of 
patient  demand  resulted  in  long  wait-times  at  the  current  OPC;  this  is  what  drives  the 
creation  of  another  OPC.  Using  DES,  Huschka,  et  al.  (2008)  suggested  clinic 
improvements  to  better  utilize  resources,  increase  patient  satisfaction,  and  more 
efficiently  use  healthcare  providers. 

DES  is  also  useful  for  studying  healthcare  facilities  larger  than  a  clinic.  Connelly 
and  Bair  (2004)  analyzed  the  average  treatment  times  patients  receive  when  checking  in 
at  an  emergency  department  at  a  hospital.  Swisher,  et  al.  (2001)  reallocated  some  of  the 
provider’s  tasks  to  a  centralized  information  center.  This  centralized  information  center 
services  a  network  of  clinics  across  the  United  States  and  its  primary  purpose  is  to  take 
administrative,  clerical,  and  scheduling  tasks  away  from  the  provider  so  the  provider  can 
focus  more  on  patient  care.  Unlike  a  system  of  clinics,  a  network  of  clinics  are 
independent  clinics  not  co-located  in  a  single  building  that  share  administrative  tasks 
through  a  centralized  function.  Because  the  scope  of  the  Swisher,  et  al.  study  focuses  on 
the  clinical  environment,  only  one  clinic  is  studied;  the  operation  of  the  network  as  a 
whole  has  yet  to  be  studied  (Swisher,  Jacobson,  Jun,  &  Balci,  2001).  The  current 
literature  on  simulation  in  healthcare  presents  a  research  opportunity  to  expand  these 
studies  beyond  the  single-clinic  level  to  a  system  of  clinics. 
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Sensitivity  Analysis  in  Healthcare 

In  any  system,  there  are  certain  properties  that  are  desired  by  the  decision  maker. 
One  such  property  that  would  be  of  interest  to  a  military  healthcare  decision  maker  is  the 
robustness  of  the  system.  There  are  various  definitions  of  robustness  (de  Week,  Ross,  & 
Rhodes,  2012;  Ryan,  Jacques,  &  Colombi,  2013).  For  this  specific  research,  robustness 
is  defined  as  “the  measure  of  how  effectively  a  system  can  maintain  a  given  set  of 
capabilities  in  response  to  external  changes  after  it  has  been  fielded  (Ryan,  Jacques,  & 
Colombi,  2013).”  In  this  case,  the  system  would  be  the  system  of  clinics  and  the  measure 
of  capability  is  the  patient  wait-time.  The  external  changes  are  the  increase  in  patient 
demand  due  to  an  increase  in  military  personnel  deployments  or  a  mild  medical  incident 
happening  on  the  military  installation.  Mild  medical  incidents  can  range  from  flu/cold 
incidents  that  do  not  require  hospital  care  to  food  poisoning  at  a  local  restaurant  that  is 
mild  enough  to  not  warrant  an  emergency  room  visit.  Simulated  experiments  of 
alternative  systems  are  used  to  evaluate  alternate  systems  being  fielded. 

To  test  the  robustness  of  a  system,  sensitivity  analysis  is  performed.  There  are 
several  studies  showing  how  sensitivity  analysis  is  used  to  test  the  robustness  of  a 
healthcare  system  (Aktas,  Ulengin,  &  Sahin,  2007;  Hashimoto  &  Bell,  1996;  Dorr,  Horn, 
&  Smout,  2005;  Doubilet,  Begg,  Weinstein,  Braun,  &  McNeil,  1985;  Angus,  Kelley, 
Schmitz,  White,  &  Popovich,  2000).  One  such  study  is  performed  by  Aktas,  Ulengin, 
and  Sahin  (2007).  Aktas,  et  al.  performed  a  sensitivity  analysis  on  a  case  study 
involving  a  private  hospital  in  Turkey.  The  tomography  section  in  the  radiology 
department  had  a  problem  with  the  process  time  because  lengthy  time  spent  on  the 
tomography  machines  have  high  operating  costs;  additional  tomography  machines  also 
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have  a  high  cost  to  purchase.  Also,  the  longer  a  doctor  spends  time  on  the  tomography 
machines,  the  more  dissatisfied  the  patients  will  be  due  to  patients  having  to  wait  in  long 
queues.  In  order  to  affect  change  in  this  system  to  reduce  patient  wait-times,  a  sensitivity 
analysis  was  conducted  to  identify  the  variables  that  are  the  most  sensitive  to  change  in 
affecting  the  process  time.  Aktas,  et  al.  found  that  the  process  time  is  very  sensitive  to 
the  process  type;  if  different  process  types  are  offered  on  different  days,  then  it  can  solve 
this  issue  with  the  exception  of  one  type.  They  found  that  the  whole  abdomen  process 
time  is  exceptionally  long  by  itself.  Additionally,  Aktas,  et  al.  (2007)  found  that  the 
process  time  is  not  sensitive  to  technicians;  improving  the  technicians  will  have  no  effect, 
if  any,  on  the  process  time. 

While  Aktas,  et  al.  (2007)  used  sensitivity  analysis  to  look  at  reducing  the  process 
time.  Dorr,  Horn,  and  Smout  (2005)  took  a  different  approach  and  used  sensitivity 
analysis  to  evaluate  the  robustness  of  cost  estimates  ranging  from  hospitalization  costs  to 
registered  nurse  cost  per  hour.  Dorr,  et  al.  (2005)  found  that  hospitalization  costs  are 
most  sensitive  to  perturbations  from  nursing  home  residents. 

Hashimoto  and  Bell  (1996)  conducted  a  sensitivity  analysis  on  clinic  staffing  to 
evaluate  the  patient  time  in  the  clinic,  session  length,  and  idle  times  of  the  doctor  for  a 
single  clinic.  Hashimoto  and  Bell  (1996)  showed  that  patient  total  time  in  the  clinic  is 
sensitive  to  number  of  doctors  in  the  clinic;  increasing  the  doctor  staffing  level  decreased 
the  average  patient  time  in  system  by  18.3  minutes. 
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Cost-Benefit  Analysis  in  Healthcare 

Comparable  to  cost-value  analysis  and  cost-effectiveness  analysis,  cost-benefit 
analysis  studies  have  also  been  performed  in  the  healthcare  field  (Brown,  Brown, 

Sharma,  &  Landy,  2003;  Nord,  1993).  The  overarching  structure  of  these  analyses  stems 
from  costs  incurred  in  relation  to  gains  realized.  Brown,  Brown,  Sharma,  and  Landy 
(2003)  define  cost-benefit  analysis  as  a  measure  of  “both  the  costs  and  the  outcomes  of 
alternative  interventions  in  terms  of  dollars  (resources).” 

Several  studies  analyze  the  relationship  of  the  costs  to  gains  in  other  healthcare 
systems  (Nord,  1993;  Eichler,  Kong,  Gerth,  Mavros,  &  Jonsson,  2004;  van  den  Bemt,  et 
al.,  2002).  Nord  (1993)  compared  the  costs  of  different  medical  interventions  to  the 
effect  treatments  have  on  patients.  This  resulted  in  giving  medical  decision  makers  the 
ability  to  prioritize  healthcare  programs  in  terms  of  cost  per  one  Saved  Young  Life 
Equivalent  (SAVE)  (Nord,  1993).  Eichler,  Kong,  Gerth,  Mavros,  and  Jonsson  (2004) 
took  a  different  approach  in  prioritizing  healthcare  resources  by  analyzing  the  cost  to  gain 
ratio  in  terms  of  an  acceptable  threshold  to  allocate  scarce  resources.  Van  den  Bemt, 
Postma,  van  Roon,  Chow,  Fijn,  and  Brouwers  (2002)  looked  into  reducing  prescription 
errors  by  hospital  pharmacy  staff  by  conducting  a  cost-benefit  analysis.  In  order  to  save 
money,  money  must  first  be  spent.  Van  den  Bemt  et  al.  (2002)  demonstrated  that  when 
the  hospital  invested  more  time  to  properly  prescribe  medication  to  patients,  a  net  cost  of 
€285  to  a  benefit  of  €9867  is  attributed  to  a  reduction  of  prescribing  errors. 
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Research  Gap 

Despite  the  growing  proliferation  of  simulation  studies  in  healthcare,  little  has 
been  done  in  using  DES  to  model  a  system  of  clinics  in  the  private  practice  and  no  study 
was  found  in  military  healthcare.  Existing  research  primarily  focuses  on  stand-alone 
clinics,  whereas  this  research  takes  a  novel  approach  of  examining  a  system  of  clinics,  in 
which  some  resources  are  shared.  Evaluating  the  individual  clinics  in  a  system  of  clinics 
does  not  reveal  how  the  performance  of  one  clinic  affects  the  performance  of  other 
clinics.  This  research  evaluates  the  impacts  of  various  staffing  levels  on  patient  wait-time 
for  the  system  of  clinics  at  Wright-Patterson  Air  Force  Base  (WPAFB),  Ohio.  This 
research  uses  DES  to  identify  the  system  of  clinic’s  bottlenecks,  assess  the  system  of 
clinic’s  overall  wait-time  and  throughput,  and  investigates  the  effects  of  altering  the 
staffing  levels.  Because  resources  are  shared  in  the  existing  system  of  clinics,  this  study 
hypothesizes  that  rebalancing  the  staffing  levels  of  individual  clinics  can  reduce  the 
average  patient  wait-time  of  the  system  of  clinics  as  a  whole. 

Despite  the  various  studies  conducted  on  sensitivity  and  cost-benefit  analysis,  no 
work  has  been  found  to  study  the  effects  of  increasing  patient  demand  have  on  a  system 
of  clinics.  In  particular,  no  study  is  found  that  analyzes  this  effect  due  to  (1)  increased 
military  deployments  or  (2)  a  surge  of  patient  “walk-ins”  due  to  a  mild  medical  incident 
in  the  local  area.  This  research  takes  a  novel  approach  of  using  both  sensitivity  analysis 
and  cost-benefit  analysis  to  evaluate  the  simulated  effects  of  an  increase  in  patient 
demand  on  a  system  of  clinics  where  some  resources  are  shared.  This  research  evaluates 
the  robustness  of  different  staffing  level  combinations  when  the  system  is  subjected  to 
deployment  demand  as  well  as  a  medical  incident  demand.  This  study  hypothesizes  that 
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scenarios  with  increased  patient  demand  will  be  most  negatively  impacted  (longer  patient 
wait-times)  for  alternative  scenario(s)  that  decrease  the  number  of  flight  medicine 
doctors. 

This  research  also  evaluates  the  savings  estimates  in  implementing  alternative 
scenarios:  where  the  staffing  levels  vary  in  the  system  of  clinics  while  maintaining  a 
zero-sum  manning  level  when  the  system  of  clinics  is  subjected  to  deployment  and 
medical  incident  demands.  This  study  hypothesizes  that  one  or  more  staffing  level  will 
have  a  statistically  lower  cost  to  implement  than  the  remaining  staffing  levels. 


Summary 

This  chapter  provided  an  overview  of  the  healthcare  industry,  simulation  in 
healthcare,  sensitivity  analysis  in  healthcare,  and  cost-benefit  analysis  in  healthcare.  This 
chapter  also  identified  the  gap  in  the  literature  that  this  research  addresses.  DES  has 
become  a  well-established  tool  for  evaluating  clinic  processes.  Sensitivity  analysis  is  a 
common  tool  to  evaluate  the  robustness  of  a  system  where  a  system  maintains  its  level  of 
performance  in  a  changing  environment  is  highly  desired.  Cost-benefit  analysis  is 
common  in  businesses,  but  healthcare  decision  makers  are  starting  to  utilize  this 
technique  in  aiding  their  decisions.  The  next  two  chapters  address  the  research  question 
through  a  conference  paper  that  answers  the  first  investigative  question  and  a  journal 
article  that  answers  the  second  and  third  investigative  questions. 
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III.  Industrial  and  Systems  Engineering  Research  Conference  Paper 


Reducing  Wait-Time  of  a  System  of  Clinics  Using  Discrete-Event  Simulation 

Michael  Q.  Corpuz,  Christina  F.  Rusnock,  Vhance  V.  Valencia,  and  Kyle  F.  Oyama 
Department  of  Systems  Engineering  and  Management 
Air  Force  Institute  of  Technology 
Wright-Patterson  AFB,  OH  45433  USA 


Abstract 

Inefficiencies  in  the  U.S.  healthcare  system  are  a  growing  concern.  Fong  wait-times  are 
of  particular  concern  for  active  duty  military  personnel,  as  long  waits  at  military  clinics 
unnecessarily  take  active  duty  personnel  away  from  performing  their  military  duties. 
Managing  wait-times  can  be  particularly  challenging  not  only  due  to  variable  patient 
demands,  but  also  due  to  variability  in  the  number  of  providers  caused  by  frequent 
relocations  of  military  personnel.  Existing  research  primarily  focuses  on  stand-alone 
clinics,  whereas  this  research  takes  the  novel  approach  of  examining  a  system  of  clinics, 
in  which  some  resources  are  shared.  This  research  evaluates  the  impacts  of  variable 
staffing  levels  on  patient  wait-time  for  a  system  of  clinics  at  Wright-Patterson  AFB,  using 
Discrete-Event  Simulation  (DES)  to  identify  bottlenecks  within  the  system  of  clinics, 
assess  the  overall  wait-time  and  throughput  of  the  system  of  clinics,  and  investigate  the 
effects  of  altering  the  staffing  levels.  This  study  finds  that  resources  are  misallocated 
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within  the  system  of  clinics,  with  too  few  resources  devoted  to  the  optometry  clinic  and 
too  many  resources  devoted  to  the  other  clinics.  To  effectively  manage  resources  and 
patient  wait-times,  this  study  recommends  a  rebalancing  of  military  manning  allocations. 


Keywords 

discrete  event  simulation;  military  healthcare;  system  of  clinics;  healthcare  staffing  levels 


1.  Introduction 

Inefficiencies  in  the  U.S.  healthcare  system  are  receiving  nationwide  public 
attention.  It  is  reported  that  healthcare  systems  waste  billions  of  dollars  [1];  even 
President  Barack  Obama  has  concerns  over  the  inefficiencies  in  the  healthcare  system 
[2].  With  variable  demand  for  healthcare  services,  inefficiencies  in  the  system, 
particularly  inefficiencies  in  military  clinics,  must  be  eliminated  in  order  to  sustain 
healthcare  services  in  the  coming  decades.  Long  wait-times  for  healthcare  are  of 
particular  concern  for  active  duty  personnel  because  the  long  wait-times  take  these 
personnel  away  from  performing  their  official  duties.  To  address  this  ongoing  issue,  this 
research  uses  discrete-event  simulation  (DES)  to  investigate  the  effects  of  changing 
staffing  levels.  Of  particular  interest  is  to  understand  the  effect  of  such  changes  on 
process  wait-times  while  maintaining  the  overall  manning  level  of  the  entire  system. 

This  study  hypothesizes  that  adjusting  staffing  levels  of  different  clinics  within  the 
system  of  clinics  will  reduce  the  wait-times  experienced  by  patients  at  various  processes 
within  the  system,  while  maintaining  an  overall  zero-sum  manning  level.  This  study 
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evaluates  a  system  of  clinics  located  on  Wright-Patterson  Air  Force  Base,  Ohio.  This 
system  of  clinics  consists  of  five  separate  clinics:  Flight  Medicine  Clinic,  Occupational 
Medicine  Clinic,  Hearing  Conservation  Clinic,  Audiology,  and  Optometry.  This  group  of 
clinics  is  considered  a  system  of  clinics  because  they  are  co-located  in  a  single  building 
sharing  staffing  and  room  resources. 


2.  Background 

DES  is  a  type  of  simulation  that  mimics  the  operation  of  a  real-world  system  at 
discrete  points  over  time  [3].  Given  its  capability,  DES  is  an  effective  tool  in  studying 
healthcare  systems  because  it  is  particularly  suited  to  analyzing  systems  with  queues, 
variable  processing  times,  and  emergent  system  behavior.  DES  can  be  used  to  analyze 
numerous  healthcare  system  problems:  reducing  patient  wait-times;  managing  utilization 
rates;  identifying  bottlenecks;  and  evaluating  alternative  system  effectiveness. 

2.1  Literature  Review 

Simulation  is  widely  used  in  the  field  of  healthcare  to  discover  potential  solutions 
to  the  issue  of  system  inefficiencies  [4-10].  For  example,  Giachetti  [4]  looks  at  using 
simulation  to  combat  long  wait-times  patients  face  for  scheduled  appointments.  Long 
wait-times  due  to  overbooking  cause  patient  dissatisfaction.  Patient  dissatisfaction 
increases  patient  no-shows  the  next  time  an  appointment  is  booked  and,  in  turn,  cause  a 
reduced  output  of  clinical  care;  patient  no-shows  are  missed  opportunities  for  other 
patients  to  see  the  doctor.  Giachetti  found  that  removing  multiple  appointment  types  can 
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reduce  patient  wait-time.  Giachetti  further  suggests  that  minimizing  patient  wait-time 
would  increase  patient  satisfaction  and  decrease  no-shows. 

DES  is  a  useful  tool  in  improving  and  establishing  individual  clinics.  Kim,  et  al. 
[5]  found  that  adding  an  additional  psychiatrist  and  extending  daily  hours  of  an  operation 
by  two  hours  can  effectively  reduce  the  service  time  by  14.6  minutes,  on  average,  in 
order  to  improve  access  to  mental  health  services  at  a  mental  health  clinic.  Cote  [6]  used 
DES  to  determine  the  capacity  of  examination  rooms  as  healthcare  demands  increase, 
finding  that  it  is  the  physician,  not  the  number  of  examination  rooms  that  influences  the 
quality  of  care.  Despite  the  physician  influencing  the  quality  of  care,  if  physicians  are 
consistently  over  utilized,  then  the  quality  of  care  diminishes.  This  is  due  to  reduced  time 
spent  by  the  physician  with  each  patient  in  order  to  meet  patient  demand  [6].  When 
physicians  are  being  rushed,  they  also  increase  the  risk  of  making  errors  which  result  in 
patient  needs  not  being  met.  Physicians  typically  belong  to  independent  clinics  where 
they  do  not  share  resources  with  other  clinics.  Huschka,  et  al.  [7]  used  DES  to  establish 
an  Outpatient  Procedure  Center  (OPC).  An  increase  of  patient  demand  resulted  in  long 
wait-times  at  the  current  OPC;  this  is  what  drives  the  creation  of  another  OPC.  Using 
DES,  Huschka,  et  al.  [7]  suggested  clinic  improvements  to  better  utilize  resources, 
increase  patient  satisfaction,  and  more  efficiently  use  healthcare  providers. 

DES  is  also  useful  for  studying  healthcare  facilities  larger  than  a  clinic.  Connelly 
and  Bair  [8]  analyzed  the  average  treatment  times  patients  receive  when  checking  in  at  an 
emergency  department  at  a  hospital.  Swisher,  et  al.  [9]  reallocated  some  of  the  provider’s 
tasks  to  a  centralized  information  center.  This  centralized  information  center  services  a 
network  of  clinics  across  the  United  States  and  its  primary  purpose  is  to  take 
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administrative,  clerical,  and  scheduling  tasks  away  from  the  provider  so  the  provider  can 
focus  more  on  patient  care.  Unlike  a  system  of  clinics,  a  network  of  clinics  are 
independent  clinics  not  co-located  in  a  single  building  that  share  administrative  tasks 
through  a  centralized  function.  Because  the  scope  of  the  Swisher,  et  al.  study  focuses  on 
the  clinical  environment,  only  one  clinic  is  studied;  the  operation  of  the  network  as  a 
whole  has  yet  to  be  studied  [9] .  The  current  literature  on  simulation  in  healthcare 
presents  a  research  opportunity  to  expand  these  studies  beyond  the  single-clinic  level  to  a 
system  of  clinics. 

2.2  Research  Gap 

Despite  the  growing  proliferation  of  simulation  studies  in  healthcare,  little  has 
been  done  in  using  DES  to  model  a  system  of  clinics  in  the  private  practice  and  no  study 
was  found  in  military  healthcare.  Existing  research  primarily  focuses  on  stand-alone 
clinics,  whereas  this  research  takes  a  novel  approach  of  examining  a  system  of  clinics,  in 
which  some  resources  are  shared.  Evaluating  the  individual  clinics  in  a  system  of  clinics 
does  not  reveal  how  the  performance  of  one  clinic  affects  the  performance  of  other 
clinics.  This  research  evaluates  the  impacts  of  various  staffing  levels  on  patient  wait-time 
for  the  system  of  clinics  at  Wright-Patterson  Air  Force  Base  (WPAFB),  Ohio.  This 
research  uses  DES  to  identify  the  system  of  clinic’s  bottlenecks,  assess  the  system  of 
clinic’s  overall  wait-time  and  throughput,  and  investigates  the  effects  of  altering  the 
staffing  levels.  Because  resources  are  shared  in  the  existing  system  of  clinics,  this  study 
hypothesizes  that  rebalancing  the  staffing  levels  of  individual  clinics  can  reduce  the 
average  patient  wait-time  of  the  system  of  clinics  as  a  whole. 
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3.  Baseline  Discrete-Event  Simulation  Model 


To  evaluate  the  hypothesis,  this  study  begins  with  the  development  of  a  baseline 
DES  model.  The  first  step  in  developing  a  baseline  model  is  to  formulate  a  conceptual 
model  of  the  system  in  order  to  ensure  that  system  tasks,  resources,  and  work  flows  are 
accurately  captured.  Next,  the  required  input  data  are  collected  and  fitted  to  probability 
distributions.  Analysis  of  the  input  data  is  combined  with  the  conceptual  model  of  the 
system  into  a  task  network  that  forms  the  baseline  simulation  model.  The  baseline 
simulation  model  features  the  task  flows,  arrival  rates,  process  probability  distributions, 
system  resources,  and  probabilistic  events.  Finally,  time  in  system  (TIS)  data  from  the 
baseline  simulation  model  are  validated  against  the  TIS  data  from  the  real  world  system. 
This  method  is  further  described  in  the  subsections  that  follow. 

3.1  Conceptual  Model 

The  first  step  in  creating  a  usable  baseline  simulation  model  is  to  understand  the 
system  of  clinics  being  studied.  In  order  to  understand  the  system  of  clinics,  a  conceptual 
model  of  daily  operations  is  developed.  To  develop  this  framework,  staff  members  of  the 
system  of  clinics  provided  a  general  description  of  daily  operations,  graphically  depicted 
in  Figure  1.  A  typical  daily  operation  starts  when  patients  check  in  at  the  front  desk  upon 
arrival.  Patients  are  given  paperwork  to  fill  out,  if  needed.  Then,  patients  visit  other 
stations  where  a  nurse  or  technician  perform  various  tasks  on  them  (e.g.,  check 
vitals/preparation,  laboratory  work,  electrocardiogram  (ECG),  X-ray,  visit  other  clinics, 
and  additional  visits  to  nurses  or  technicians)  if  they  are  required  prior  to  visiting  the 
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doctor.  After  these  preparatory  tasks,  patients  wait  until  the  doctor  is  available.  When 
patients  are  waiting,  the  current  queuing  strategy  of  the  system  of  clinics  at  WPAFB  is  a 
priority  queue:  Patients  with  a  scheduled  appointment  have  priority  to  see  the  doctor 
over  walk-in  patients.  After  visiting  with  the  doctor,  a  follow-up  appointment  is 
scheduled  if  an  additional  visit  is  required.  Patients  then  exit  the  system  of  clinics. 


Figure  1:  System  of  Clinics  Task  Network 
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3.2  Data  Collection 


For  each  activity  described  in  the  conceptual  model,  timing  and  decision  data  are 
required  in  order  to  build  the  simulation  model.  These  data  were  collected  by  the  clinic 
medical  staff  by  first  performing  a  trial  data  collection  effort  during  the  month  of  July 
2014  to  become  familiar  with  the  data  collection  process.  The  official  data  collection 
effort  was  conducted  in  August  2014.  The  start  and  end  times  for  each  process  were 
collected  using  a  clipboard  with  an  integrated  clock  and  a  data  collection  sheet.  The  data 
collection  sheet  records  the  following  general  information  about  the  patient’s  visit:  clinic 
type  (audiology,  flight  medicine  clinic,  hearing  conservation  clinic,  occupational 
medicine  clinic,  or  optometry),  patient  type  (military,  civilian,  or  dependent),  status 
(scheduled  appointment  or  walk-in),  date,  appointment  time  (if  applicable),  and 
appointment  type.  The  data  collection  sheet  also  records  the  start  and  end  time  of  each 
process  the  patient  undergoes  which  includes:  patient  check-in,  filling  out  paperwork, 
hearing  conservation  visit,  checking  vitals/preparing  patient,  nurse  or  technician  visit, 
laboratory  tasks,  X-ray  examination,  ECG,  provider  visit,  additional  provider  visits,  and 
scheduling  a  follow-up  appointment.  Annotating  the  start  and  end  times  on  the  sheet 
have  negligible  impact  on  the  performance  of  the  medical  staffs  duties.  A  few  of  the 
processes  were  performed  infrequently,  thus  failing  to  provide  an  adequate  number  of 
observations  during  the  August  2014  collection  period.  Thus,  the  data  collection  effort 
was  extended  to  include  the  trial  data  from  July  2014  and  an  additional  collection  from 
September  2014  for  these  infrequent  tasks:  laboratory  tasks,  X-ray  examination,  ECG, 
and  visits  to  additional  nurses  or  technicians.  It  is  reasonable  to  include  some  data  from 
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the  trial  period  in  July  2014  for  these  processes  because  the  times  for  these  tasks  were 
accurately  collected. 

3.3  Input  Analysis 

Upon  completion  of  the  data  collection  effort,  input  data  modeling  was  performed 
on  the  patient  arrivals  and  process  times  in  order  to  form  probability  distributions.  These 
probability  distributions  were  tested  for  independence,  homogeneity,  and  goodness-of-fit. 
All  of  the  final  distributions  in  the  baseline  model  either  successfully  passed  these  tests  or 
were  replaced  by  an  empirical  distribution  directly  representing  the  data.  Table  1 
summarizes  the  frequency  for  each  process  and  possible  patient  path  flows  within  the 
system  of  clinics;  this  information  is  used  to  establish  the  decision  logic  for  the 
simulation  model.  Table  2  summarizes  the  frequency  counts  for  the  clinic  visited,  patient 
type,  and  status  of  the  patient;  these  frequencies  are  used  to  establish  the  decision  logic 
for  the  simulation  model.  Table  3  summarizes  the  likelihood  of  an  optometry  patient 
seeing  the  optometrist  twice  in  a  single  visit;  this  is  unique  from  other  processes  in  that 
the  patient  visits  the  optometrist  again  whereas  the  patient  visits  other  processes  only 
once.  Table  3  is  used  to  establish  the  decision  logic  for  the  simulation  model.  Table  4 
summarizes  the  probability  distribution  for  each  of  the  datasets  being  fitted;  these 
distributions  are  used  in  the  simulation  model  to  determine  inter-arrival  times  for  patients 
entering  the  system,  as  well  as  process  times  for  each  process  visited  by  a  patient  as  they 
go  through  the  system. 
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Table  1:  Flight/Occupational  Medicine  Clinic  Process  Frequency  Counts 


Assign  Flight/Occupational  Medicine  Paperwork 


Paperwork  Count  Total  Count  Paperwork 

No  Paperwork  1 

Flight  Medicine 
Occupational  Medicine 

98 

104 

94% 

6% 

95 

179 

53% 

47% 

Need  to  Visit  Hearing  Conservation?  (For  Flight/Occupationa  Medicine  Patients) 


Obs  Count 

Total  Count 

TRUE 

FALSE  ) 

Fligl 

ht  Medicine 

Occupa' 

tional  Medicin 

Need  to  Visit  Nurse  or  Tech  2nd  Time?  (For  Flight/Occupationa  Medicine  Patients) 


1  Obs  Count 

Total  Count 

TRUE 

FALSE  1 

Fligl 

ht  Medicine 

1  i 

104 

1% 

99% 

Occupa' 

tional  Medicin 

1 _ ? _ 

179 

2% 

98% 

Need  to  Visit  Nurse  or  Tech  3rd  Time?  (For  Flight/Occupationa  Medicine  Patients) 


Need  to  See  Optometrist?  (For  Flight/Occupationa  Medicine  Patients) 


1 

1  Obs  Count 

Total  Count 

TRUE 

FALSE  1 

|  See  Optometrist 

1  i 

283 

|  2% 

98%  | 

Need  Follow  Up  Appointment?  (For  Flight/Occupationa  Medicine  Patients) 


Obs  Count 

Total  Count 

TRUE 

FALSE 

1  Schedule  Follow  Up 

17 

283 

6% 

Need  to  Visit  Lab  or  ECG  or  X  Ray? 

Also  Need  ECG  or  X  Ray?  (After  Lab  Assigned) 

1 

Count 

TRUE 

Count 

TRUE 

Lab 

18 

6% 

- ► 

ECG 

2 

11% 

ECG 

2 

1% 

\ 

X  Ray 

1 

6% 

X  Ray 

13 

5% 

\ 

No  Visit  Needed 

15 

No  Visit  Needed 

250 

88% 

\ 

\  Also  Need  X  Ray?  (After  ECG  Assigned) 

* 

Count  TRUE  1 

X  Ray 

1  50% 

No  Visit  Needed 

1  50% 

Table  2:  Patient  Attribute  Frequency  Counts 


Clinic  Visited 

Count 

Percent  1 

Audiology 

68 

13% 

Flight  Medicine 

104 

20% 

Hearing  Conservation 

37 

7% 

Occupational  Medicine 

WE 1 

Optometry 

PatientType  f 

Count 

Percent  1 

Civilian  Employee 

177 

34% 

Dependent 

88 

17% 

Military 

252 

49% 

Status 

Count 

Percent  1 

Scheduled  Appointment 

474 

91% 

Walk  In 

45 

9% 

Table  3:  Optometry  Clinic  Frequency  Count 


See  Optometrist  Twice? 


18 

133 

14% 

86% 
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Table  4:  Probability  Distribution  Summary  Table  of  Inter- Arrival/Process  Times  (in 


seconds) 


Create/Process  Node 

Distribution 

Parameters 

K-S  Test 
p- value 

Sample 

Mean 

Sample 
Std.  Dev. 

Arrive  System  of  Clinics 

Weibull 

k  =  0.778 

Lambda  =844 

>0.15 

994 

1550 

Check  In 

Empirical 

N/A 

N/A 

20 

26 

Prepare  to  See  Optometrist  with  Nurse  or  Tech 

Erlang 

ExpMean  =  292 
k  (int)  =  2 

0.0606 

599 

488 

Visit  Optometrist 

Erlang 

ExpMean  =  790 
k  (int)  =  2 

0.0604 

1700 

1100 

Dilation  Effect  Delay 

Weibull 

k  =  0.616 

Lambda  =  1320 

>0.15 

2020 

2680 

Visit  Audiologist 

Weibull 

k  =  1.34 

Lambda  =  1220 

0.131 

1870 

833 

Visit  Hearing  Conservation  Technician 
(Hearing  Conservation  Clinic  Only) 

Exponential 

Mean  =  532 

>0.15 

1080 

495 

Visit  Hearing  Conservation  Technician 
(Non  Hearing  Conservation  Clinics) 

Beta 

Alphal  =  1.79 
Alpha2  =  5.2 

0.136 

861 

328 

Fill  Flight  Medicine  Paperwork 

Weibull 

k  =  1.71 

Lambda  =  311 

>0.15 

289 

155 

Fill  Occupational  Medicine  Paperwork 

Exponential 

Mean  = 454 

>0.15 

463 

472 

Check  Vitals 

Gamma 

Alpha  =  575 
Beta  =  1.34 

0.113 

822 

722 

Perform  Lab 

Erlang 

ExpMean  =  261 
k  (int)  =  2 

0.119 

574 

321 

Perform  ECG 

Exponential 

Mean  =  384 

>0.15 

577 

408 

Perform  X  Ray 

Beta 

Alphal  =  0.926 
Alpha2  =  2.28 

>0.15 

817 

492 

2nd  Session  with  Flight  Medicine  Nurse  or  Tech 

N/A 

Constant  =  377 

N/A 

N/A 

N/A 

2nd  Session  with  Occupational  Medicine  Nurse  or  Tech 

Exponential 

Mean  =  68.8 

>0.15 

548 

90.6 

3rd  Session  with  Flight  Medicine  Nurse  orTech 

N/A 

Constant  =  634 

N/A 

N/A 

N/A 

3rd  Session  with  Occupational  Medicine  Nurse  orTech 

N/A 

Constant  =  818 

N/A 

N/A 

N/A 

See  Flight  Medicine  Physician 

Weibull 

k  =  1.19 

Lambda  =  1170 

>0.15 

1260 

917 

See  Occupational  Medicine  Physician 

Weibull 

k  =  1.49 

Lambda  =801 

>0.15 

984 

480 

Make  Follow  Up  Appointment 

Weibull 

k  =  0.595 

Lambda  =  130 

>0.15 

171 

188 

3.4  Arena  Model 

The  input  data  described  above  are  combined  with  the  process  flows  to  form  a 
task  network.  Figure  1  in  Section  3.1  provides  a  task  network  which  is  a  visual 
representation  of  the  conceptual  model.  The  conceptual  model  is  translated  into  a  task 
network  by  representing  decision  logic  as  decision  nodes,  processes  as  task  nodes,  and 
the  order  of  these  tasks  and  decisions  as  directional  arcs.  When  a  patient  goes  through 


27 


the  system,  the  individual  will  process  through  the  various  nodes  established  in  the  task 
network  and  will  follow  the  decision  logic  throughout  the  model.  For  example,  all 
patients  go  through  the  check-in  node,  followed  by  decision  logic  to  determine  (1)  if  the 
patient  needs  to  fill  out  paperwork  and  (2)  which  clinic  the  patient  will  visit: 
Audiology/Hearing  Conservation  Clinics,  Optometry  Clinic,  or  Flight/Occupational 
Medicine  Clinics. 

If  the  clinic  to  be  visited  is  Audiology  or  Hearing  Conservation,  the  task  flow  is 
simple.  The  patient  will  go  to  the  respective  node  of  Visit  Doctor/Specialist  to  process 
through  a  visit  with  the  audiologist  or  hearing  conservation  technician.  Once  completed, 
the  patient  exits  the  system. 

If  the  clinic  to  be  visited  is  Optometry,  then  the  optometry  nurse  or  technician 
prepares  the  patient  to  see  the  optometrist.  The  patient  then  visits  the  optometrist. 
Decision  logic  is  used  to  determine  if  the  patient  needs  to  see  the  optometrist  again.  If 
the  patient  needs  to  see  the  optometrist  again,  it  is  because  their  eyes  need  to  be  dilated 
for  examination.  The  patient  waits  for  the  dilation  drug  to  take  effect  before  visiting  the 
optometrist  a  second  time;  the  patient  waiting  for  the  dilation  drug  to  take  effect  is 
counted  as  value-added  time  and  not  attributed  to  wait-time  because  this  is  a  necessary 
process.  Once  completed,  the  patient  exits  the  system. 

If  the  clinic  to  be  visited  is  Flight  Medicine  or  Occupational  Medicine,  the  nurse 
or  technician  of  their  respective  clinic  checks  the  vitals  of  the  patient.  A  series  of 
decision  nodes  is  created  to  determine  if  the  patient  needs  to  perform  various  tasks.  If  the 
patient  needs  to  perform  a  laboratory  task,  ECG,  X-ray,  see  a  nurse  or  technician 
numerous  times,  visit  hearing  conservation  technician,  and/or  visit  the  optometrist  once, 
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then  the  patient  will  go  to  the  needed  process  nodes  in  any  order  based  upon  availability. 
If  the  process  node  is  using  up  all  the  resources  to  perform  the  task,  then  the  patient 
proceeds  to  the  next  task  and  returns  to  the  previous  node  when  it  becomes  available. 
When  completed,  the  patient  will  visit  the  doctor  of  the  respective  clinic.  A  decision 
node  determines  if  the  patient  makes  a  follow  up  appointment.  Once  completed,  the 
patient  exits  the  system. 

In  addition  to  capturing  the  process  flows,  decision  logic,  and  timing  data,  the  clinic 
staff  also  annotated  the  type  and  quantity  of  resources  used.  Table  5  summarizes  the 
resources  used  in  the  system  of  clinics.  There  are  unique  characteristics  associated  with  a 
few  of  the  resources.  These  characteristics  are  listed  here: 

•  There  is  a  front  desk  station  at  the  entrance  of  the  building  that  can  only  be 
manned  by  one  administration  technician. 

•  The  Hearing  Conservation  technician  is  being  treated  as  a  provider  for  this  clinic. 

•  The  Flight  Medicine  and  Occupational  Medicine  clinics  share  6  examination 
rooms. 

•  The  laboratory  and  ECG  rooms  are  operated  by  the  nurse  or  technicians  of  either 
the  Flight  Medicine  or  Occupational  Medicine  clinic,  depending  on  which  clinic 
the  patient  belongs  to. 

•  The  ECG  room  is  co-located  with  one  of  the  optometry  examination  rooms. 

•  The  X-Ray  room  is  manned  by  an  X-Ray  technician. 

Once  the  resources  were  incorporated  into  the  model,  it  was  then  validated. 
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Table  5:  System  of  Clinic  Resources 


Rooms 

Providers 

Nurse/Tech 

Front  Desk  Station 

1 

Not  Applicable 

1 

Audiology 

3 

2 

0 

Flight  Medicine 

shared  6  with  occupational  medicine 

4 

shared  8  with  Lab  and  ECG 

Hearing  Conservation 

1 

1 

Not  Applicable 

Occupational  Medicine 

shared  6  with  flight  medicine 

4 

shared  4  with  Lab  and  ECG 

Optometry 

1  dedicated  to  optometry  with  1  shared  with  ECG 

1 

1 

Lab 

2 

Not  Applicable 

performed  by  respective  clinic's  nurse/tech 

ECG 

1  shared  with  optometry 

Not  Applicable 

performed  by  respective  clinic's  nurse/tech 

X  Ray 

1 

Not  Applicable 

1 

3.5  Validation 

Validation  is  an  important  step  in  creating  a  baseline  simulation  model.  It 
provides  statistical  evidence  that  the  model  adequately  reflects  the  real  world  system.  For 
satisfactory  validation,  a  confidence  interval  range  that  is  within  10%  above  and  below 
the  mean  is  desired.  For  this  system,  the  average  time  in  system  is  54.13  minutes,  thus  a 
half- width  of  5.4  min  or  less  is  required.  A  99%  confidence  interval  for  this  system 
produced  a  half- width  of  3.16  minutes,  thus  a  99%  confidence  interval  level  was  deemed 
sufficient  for  use  in  validation.  A  tradeoff  in  using  a  99%  confidence  interval  level  is 
that,  although  it  provides  a  high  level  of  confidence,  it  does  so  at  the  risk  of  an 
unacceptably  large  half-width.  Because  the  99%  confidence  interval  level  has  a  half¬ 
width  that  is  considerably  less  than  the  desired  ±10%  of  the  mean,  the  99%  confidence 
interval’s  half-width  is  deemed  to  be  acceptably  narrow. 

Upon  establishing  a  confidence  interval  level,  real  world  data  is  compared  to 
simulation  data  using  a  99%  confidence  interval.  In  order  to  determine  the  number  of 
replications  needed  to  run  the  model,  an  approximation  equation  is  used  [11]: 


h2 

a0 

h 2 


(1) 
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Where  n  is  the  number  of  replications  needed,  n0  is  the  number  of  replications  in  the 
initial  production  run,  h0  is  the  half-width  of  the  initial  production  run,  and  h  is  the 
desired  half-width.  An  initial  run  on  the  model  is  conducted  with  n0=10  as  an  arbitrary 
initial  number  of  replications.  It  produces  an  initial  half-width  of  h0  =  6.03  min.  Based 
on  the  desired  half- width  of  h  —  3.16  min  (taken  from  the  real  world  half-width  of  3.16 
minutes),  an  estimate  on  the  number  of  replications  needed,  n,  is  evaluated;  first  iteration: 
n=10(6.032/3.162)  =  36.49.  This  process  is  repeated  three  more  times  to  determine  a 
reasonable  number  of  replications;  second  iteration:  37(3.3 12/3.162)  =  40.68;  third 
iteration:  41(3.21 2/3 . 1 62)  =  42.39;  fourth  iteration:  43(3.092/3.162)  =  41.20.  It  is 
determined  that  at  least  41.20,  rounded  up  (to  be  conservative)  to  42  replications,  is 
required  to  achieve  the  desired  half-width.  The  confidence  intervals  of  the  real  world 
data  and  simulation  data  reveal  that  there  is  no  statistical  difference  between  the  model 
and  the  real  world  system,  see  Figure  2.  This  is  demonstrated  by  the  overlap  of  both 
confidence  intervals,  thus  validating  the  baseline  model.  Note  that  the  average  time  in 
system  of  the  simulation  is  49.29  minutes.  This  indicates  that  the  simulation  is,  on 
average,  4.84  minutes  faster  than  reality.  To  account  for  this  difference,  it  is 
hypothesized  that  the  exclusion  of  transit  time  in  the  model  is  what  is  causing  a  slightly 
faster  time  in  system.  This  is  of  negligible  concern  for  the  purpose  of  this  study  as  the 
patient  wait-time  is  the  focus  of  this  investigation,  not  the  total  time  in  system,  and  transit 
time  is  not  expected  to  impact  wait-time. 
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45  50  55  60 

Time  in  System  (Minutes) 


Figure  2:  Real-World  versus  Simulation  TIS  with  a  99%  Confidence  Interval 


3.6  Average  Wait  Time  Analysis  of  Individual  Processes 

After  the  baseline  model  is  validated,  the  average  wait-time  associated  with  each 
individual  process  is  analyzed  in  order  to  identify  potential  bottlenecks  in  the  system. 
Potential  bottlenecks  are  processes  that  have  a  significant  average  wait-time.  In  this  case, 
all  processes  had  an  average  wait-time  that  was  less  than  2  minutes  except  “Visit 
Optometrist”.  The  “Visit  Optometry”  process  has  an  average  wait-time  of  18.17  minutes 
and  is  clearly  the  bottleneck  in  the  system.  Figure  3  further  shows  that  the  Optometrist  is 
the  highest  utilized  resource  in  the  system;  it  is  currently  utilized  at  50.9%.  In  this  case, 
utilization  rate  is  equal  to  the  time  spent  with  patient  divided  by  time  available.  The 
utilization  rates  in  Figure  3  only  include  data  from  patient  interactions  and  thus  do  not 
include  information  about  additional  tasks  performed  by  the  clinic  staff  that  do  not 
involve  the  patient.  It  is  hypothesized  that  adding  an  optometrist  to  the  staff  will  greatly 
reduce  the  average  wait-time  of  that  process.  However,  military  units  often  struggle  to 
gain  additional  staffing  positions.  Instead,  this  study  proposes  recoding  one  of  the 
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currently  under-utilized  positions  to  change  it  into  an  optometrist  position.  This  provides 
a  zero-sum  manning  level  to  the  system  of  clinics  that  essentially  removes  a  staff  member 
from  one  of  the  clinics  and  adds  an  additional  optometrist  to  the  optometry  clinic.  This 
research  assumes  that  this  under-representation  of  optometrists  is  local  to  this  clinic  and 
does  not  indicate  a  shortage  of  optometrists  in  the  U.S.  military  medical  service.  Thus, 
the  additional  optometrist  comes  from  an  optometry  clinic  from  a  different  military 
installation  (presumably  one  that  has  too  many  optometrists).  Potential  shortages  to 
particular  medical  staffing  types  is  outside  the  scope  of  this  research,  but  is  worthy  of 


further  investigation. 
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Figure  3:  Baseline  Scenario  Resource  Utilization  Rates 


4.  Alternative  Systems 

To  implement  this  zero-sum  manning  level  recoding,  the  validated  baseline 
simulation  model  is  modified  to  incorporate  varying  staffing  levels  from  each  of  the 
clinics  in  order  to  determine  which  position  recoding  will  minimize  the  average  wait-time 
in  the  system  of  clinics.  The  staffing  levels  serve  as  the  independent  variables  and  the 
patient  wait-time  serves  as  the  dependent  variable  for  this  experiment.  The  experimental 
design  is  described  in  greater  detail  below. 
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4.1  Experimental  Design/Methodology 

Table  6  summarizes  the  simulation  experiments  that  are  conducted.  In  the  table, 

“BS”  represents  the  baseline  scenario  and  “AS”  represents  alternative  scenarios.  There 
are  five  distinct  alternative  scenarios.  Because  the  staffing  positions  are  being  recoded  to 
bring  in  an  additional  optometrist,  the  optometrist  value  is  increased  by  one  optometrist 
to  two  optometrists  in  all  alternative  scenarios  (highlighted  in  green).  While  the 
optometrist  is  increased  by  one,  a  different  staff  type  must  be  reduced  by  one  to  maintain 
a  zero-sum  manning  level.  The  hearing  conservation  technician,  optometry  nurse  or 
technician  and  the  x-ray  technician  cannot  be  reduced  because  all  staff  types  must  be 
manned  by  at  least  one  person.  There  are  five  staff  types  that  can  potentially  be  reduced 
(highlighted  in  red).  A  reduction  of  a  staff  member  from  each  feasible  staff  type 
constitutes  an  alternative  scenario: 

•  AS1  reduces  the  audiologist  by  one  and  incrementing  the  optometrist  by  one 
while  holding  all  else  constant; 

•  AS2  reduces  the  flight  medicine  doctor  by  one  and  incrementing  the  optometrist 
by  one  while  holding  all  else  constant; 

•  AS3  reduces  the  flight  medicine  nurse  or  technician  by  one  and  incrementing  the 
optometrist  by  one  while  holding  all  else  constant; 

•  AS4  reduces  the  occupational  medicine  doctor  by  one  and  incrementing  the 
optometrist  by  one  while  holding  all  else  constant; 

•  and  AS5  reduces  the  occupational  medicine  nurse  or  technician  by  one  and 
incrementing  the  optometrist  by  one  while  holding  all  else  constant. 

Each  alternative  scenario  is  a  separate  model,  and  each  model  is  run  individually,  using 

the  pre-determined  sample  size  of  42  replications. 
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Table  6:  Experimental  Design 


STAFF  TYPE 

BS 

AS1 

AS2 

AS3 

AS4 

AS5 

Audiologist 

2  ; 

2 

2 

2 

2 

Flight  Medicine  Doctor 

4 

4 

4 

4 

4 

Flight  Medicine  Nurse  orTechnician 

8 

8 

8 

8 

8 

Flearing  Conservation  Technician 

1 

1 

1 

1 

1 

1 

Occupational  Medicine  Doctor 

4 

4 

4 

4 

4 

Occupational  Medicine  Nurse  orTechnician 

4 

4 

4 

4 

4 

2 

Optometrist 

1 

2 

2 

2 

2 

Optometry  Nurse  orTechnician 

1 

1 

1 

1 

1 

1 

X  Ray  Technician 

1 

1 

1 

1 

1 

1 

5.  Analysis  and  Results 

The  output  data  from  the  5  alternative  scenarios  are  presented  in  Table  7  and 
Table  8.  Table  7  shows  the  average  wait-time,  along  with  a  95%  confidence  level  half¬ 
width,  of  the  individual  processes  the  patients  go  through  in  the  system  of  clinics.  For 
example,  when  a  patient  visits  the  optometrist,  they  have  to  wait  18.17  minutes,  on 
average,  before  seeing  the  optometrist.  Table  8  shows  the  average  total  wait-time,  along 
with  a  95%  confidence  level  half- width,  of  the  patients  when  looked  at  from  the  system 
of  clinics  perspective.  For  example,  in  the  baseline  scenario,  patients  wait  7.49  minutes, 
on  average,  when  they  process  through  the  entire  system.  Figure  4  is  a  statistical  analysis 
(two-sample  t-test)  that  tests  if  the  mean  of  the  baseline  scenario  is  significantly  different 
from  the  mean  of  each  of  the  alternative  scenarios.  If  the  value  of  zero  is  included  in  the 
range,  then  this  indicates  that  the  particular  alternative  scenario  being  analyzed  has  a 
statistically  similar  mean  to  the  baseline  scenario.  Figure  4  shows  that  the  mean  of  all 
alternative  scenarios  are  statistically  significant  from  the  baseline  scenario  with  a  95% 
level  of  confidence.  Further  investigation  shows  that  reducing  the  audiologist  staffing  by 
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one  (AS1)  would  increase  the  average  wait-time  for  the  audiologist  visit  to  4.3  minutes, 


compared  to  .3  minutes  in  the  baseline  scenario  (see  “Visit  Audiologist,”  Table  7).  AS2, 
AS3,  AS4,  and  AS5  show  little  impact  on  the  reduction  of  their  respective  staff  type  to 
the  average  wait-time  of  the  respective  processes.  AS2  and  AS4  would  be  recommended 
solutions  in  reducing  inefficiencies  in  the  system  of  clinics  since  the  cost  savings  by 
reducing  these  staff  types  from  the  payroll  is  significantly  higher  than  a  nurse  or 
technician.  Further  analysis  of  AS2  indicates  that  reducing  the  flight  medicine  doctor  by 
one  has  little  impact  on  its  utilization  rate  from  the  baseline;  the  utilization  rate  rose  from 
6.2%  to  8.8%,  see  Figure  5.  Similarly,  further  analysis  of  AS4  indicates  that  reducing  the 
occupational  medicine  doctor  by  one  has  little  impact  on  its  utilization  rate  from  the 
baseline;  the  utilization  rate  rose  from  8.7%  to  11.4%,  see  Figure  6. 


Table  7:  Average  Wait-Time  of  the  Individual  Processes  for  All  Scenarios 


Process  Node  with  Queues -Average  Wait-Time  in  Minutes  (Half-Width) 

BS 

ASl 

AS2 

AS  3 

AS4 

AS5 

2nd  Session  with  Flight  Medicine  Nurse  or  Tech 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

2nd  Session  with  Occupational  Medicine  Nurse  or  Tech 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

3rd  Session  with  Occupational  Medicine  Nurse  or  Tech 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

Check  In 

0.12(0.14) 

0.14(0.15) 

0.13(0.15) 

0.13(0.15) 

0.13(0.15) 

0.13(0.15) 

Check  Vitals_Flight  Medicine 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

Check  Vitals_Occupational  Medicine 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

Make  Follow  Up  Appointment 

0.17(0.34) 

0.18(0.34) 

0.18(0.34) 

0.18(0.34) 

0.18(0.34) 

0.18(0.34) 

Perform  ECG_Flight  Medicine 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

Perform  ECG_Occupational  Medicine 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

Perform  Lab_Flight  Medicine 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

Perform  Lab_Occupational  Medicine 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

Perform  X  Ray 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

Prepare  to  See  Optometrist  with  Nurse  or  Tech 

1.2  (0.62) 

1.32(0.56) 

1.33  (0.56) 

1.33(0.56) 

1.37(0.55) 

1.34(0.56) 

See  Flight  Medicine  Physician 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

See  Occupational  Medicine  Physician 

0(0) 

0(0) 

0(0) 

0(0) 

0.03  (0.04) 

0(0) 

Visit  Audiologist 

0.33  (0.34) 

4.3  (2.24) 

0.29  (0.3) 

0.29  (0.3) 

0.29  (0.3) 

0.29  (0.3) 

Visit  Hearing  Conservation  Technician  (Hearing  Conservation  Clinic  Only) 

1.46(1.01) 

1.23  (0.72) 

1.24(0.73) 

1.24(0.73) 

1.39  (0.77) 

1.58(0.96) 

Visit  Hearing  Conservation  Technician(Non  Hearing  Conservation  Clinics) 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

Visit  Optometrist 

18.17(7.52) 

1.77  (0.99) 

1.58  (0.99) 

1.58(0.99) 

1.83(1.06) 

1.57(0.99) 
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Utilization  Rate 


Table  8:  Average  Total  Wait-Time  for  All  Scenarios 


BS 

AS1 

AS  2 

AS3 

AS4 

AS  5 

System  of  Clinics -Average  Total  Wait  lime  in  Minutes  (Half-Width) 

7.49  (3.3) 

2.29  (0.71) 

1.4(0.53) 

1.4(0.53) 

1.51  (0.55) 

1.41  (0.54) 

Two-sample-t  Comparison  of  Means 
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Figure  4:  Two-Sample  t-Test  Comparison  of  the  Difference  between  Baseline  System 

and  Each  Alternative  System 
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Figure  6:  Alternative  Scenario  4  Resource  Utilization  Rates 


6.  Conclusion 

The  results  of  the  simulation  experiments  indicate  that  adding  an  additional 
optometrist  can  reduce  the  average  wait-time  of  the  optometry  clinic  and  the  total  wait¬ 
time  for  the  system  of  clinics.  Adding  one  more  optometrist  can  reduce  the  wait-time 
patients  visiting  the  optometry  clinic  in  a  system  of  clinics  by  as  much  as  16.6  minutes, 
on  average.  In  a  military  setting,  that  is  16.6  minutes  of  time  that  the  patients  could  be 
using  to  perform  official  military  duties. 

Due  to  difficulties  in  increasing  military  unit  manpower  requirements,  this  study 
investigated  alternatives  that  achieved  zero-sum  manning  for  the  system  of  clinics. 
Simulation  of  five  staffing  level  changes  reveals  that  there  are  four  options  that  equally 
reduce  total  average  wait-time  in  the  system  with  insignificant  impacts  to  the  individual 
process  average  wait-times.  These  options  are  reducing  a  flight  medicine  doctor,  flight 
medicine  nurse/technician,  occupational  medicine  doctor,  or  occupational  medicine 
nurse/technician.  Reducing  a  flight  medicine  doctor  or  an  occupational  medicine  doctor 
would  be  the  most  effective  in  reducing  cost  since  these  staffing  types  cost  the  most  to  be 
kept  on  the  payroll. 
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Future  work  includes  developing  other  alternative  scenarios  that  can  reduce  the 


average  wait-time.  For  example,  reducing  the  average  wait-time  could  be  achieved  by 
increasing  the  number  of  examination  rooms  available,  by  alternating  the  process  flow, 
by  changing  the  current  queuing  strategy  of  a  priority  queue  to  a  first  come  first  served 
queue,  or  by  changing  the  appointment  scheduling  process.  Additional  future  work 
includes  adding  or  removing  more  staff,  performing  the  same  analysis  on  other  military 
installations,  and  doing  an  Air  Force-wide  assessment  to  determine  if  optometry  has  a 
career  field  shortage. 
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Abstract 

Unexpected  deployments  are  a  normal  part  of  military  operations.  When  a  crisis  arises, 
military  personnel  must  be  ready  to  deploy  at  a  moment’s  notice.  For  military  readiness 
to  be  at  100%,  military  clinics  must  be  capable  of  meeting  the  surge  in  demand  when 
military  personnel  require  a  Deployment  Health  Assessment  prior  to  deploying. 

Not  only  do  military  clinics  need  to  meet  the  demand  of  deploying  military  personnel, 
they  also  need  to  be  capable  of  handling  a  surge  of  walk-ins  when  a  mild  medical 
incident  occurs  on  the  military  installation.  Additionally,  recent  fiscal  environments  of 
austere  government  funding  raise  another  issue  to  consider  when  evaluating  healthcare 
demand.  This  research  evaluates  the  impacts  of  various  staffing  levels  on  patient  wait¬ 
time  while  maintaining  a  zero-sum  manning  level  for  the  system  of  clinics  at  Wright- 
Patterson  AFB  when  there  is  a  200%  surge  in  patient  demand  due  to  mass  deployments. 
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This  evaluation  is  conducted  using  discrete-event  simulation  to  estimate  patient  wait- 


times  and  performs  a  sensitivity  analysis  of  the  increased  patient  demand.  This  research 
also  evaluates  the  cost  associated  with  changing  staffing  levels  using  a  cost-benefit 
analysis  to  determine  the  most  cost-effective  alternative  scenario.  It  is  found  that  the 
average  total  wait-time  of  alternative  systems  is  sensitive  to  deployment  demand  and 
medical  incident  demand.  Hiring  an  optometrist  while  removing  an  occupational 
medicine  doctor  provides  the  lowest  cost  to  implement  in  baseline,  deployment,  and 
medical  incident  demand  environments. 


Keywords 

discrete  event  simulation;  sensitivity  analysis;  cost  benefit  analysis;  military  healthcare; 
system  of  clinics 


1.  Introduction 

Due  to  the  uncertainty  of  future  crises,  military  service  members  can  expect  short- 
notice  deployments.  Ideally,  military  readiness  needs  to  be  at  100%  at  all  times. 
Unfortunately,  the  military  currently  faces  a  fiscally  austere  environment.  The  military’s 
budget  is  being  reduced  by  billions  of  dollars  over  the  next  two  years  and  military 
personnel  end  strength  reduced  by  tens  of  thousands  (Simeone,  2014).  If  a  crisis  occurs 
requiring  military  personnel  to  deploy,  then  military  healthcare  clinics  must  be  capable  of 
medically  clearing  personnel  for  deployment  in  a  timely  fashion. 
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This  research  studies  the  impact  of  a  200%  increase  (54  additional  patients)  in 
patient  demand  due  to  increased  deployments  (deployment  demand)  as  well  as  a  surge  of 
the  same  level  of  54  additional  patients  walking  in  during  the  first  3  hours  due  to  a  mild 
medical  incident  on  a  military  installation  (medical  incident  demand)  on  a  system  of 
clinics  located  at  Wright-Patterson  Air  Force  Base.  The  study  is  carried  out  using  a 
discrete-event  simulation  model  to  perform  a  sensitivity  analysis.  This  research  also 
conducts  a  cost-benefit  analysis  on  the  patient  wait-time  and  costs  associated  with 
varying  staffing  levels  while  maintaining  a  zero-sum  staffing  level.  This  cost-benefit 
analysis  is  conducted  with  the  same  patient  demand  scenarios  used  in  the  sensitivity 
analysis. 

Given  a  surge  in  patient  demand  (either  deployment  demand  or  medical  incident 
demand),  this  study  hypothesizes  that  one  or  more  staffing  levels  will  have  statistically 
lower  wait-times  than  other  staffing  level  scenarios.  This  study  also  hypothesizes  that 
one  or  more  staffing  levels  will  have  a  statistically  lower  cost  to  implement  than  the  other 
staffing  level  scenarios  on  all  three  conditions  of  the  patient  demand  (baseline  demand, 
deployment  demand,  and  medical  incident  demand). 


2.  Literature  Review 

In  any  system,  there  are  certain  properties  that  are  desired  by  the  decision  maker. 
One  such  property  that  would  be  of  interest  to  a  military  healthcare  decision  maker  is  the 
robustness  of  the  system.  There  are  various  definitions  of  robustness  (de  Week,  Ross,  & 
Rhodes,  2012;  Ryan,  Jacques,  &  Colombi,  2013).  For  this  specific  research,  robustness 
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is  defined  as  “the  measure  of  how  effectively  a  system  can  maintain  a  given  set  of 
capabilities  in  response  to  external  changes  after  it  has  been  fielded  (Ryan,  Jacques,  & 
Colombi,  2013).”  In  this  case,  the  system  would  be  the  system  of  clinics  and  the  measure 
of  capability  is  the  patient  wait-time.  The  external  changes  are  the  increase  in  patient 
demand  due  to  an  increase  in  military  personnel  deployments  or  a  mild  medical  incident 
happening  on  the  military  installation.  Mild  medical  incidents  can  range  from  flu/cold 
incidents  that  do  not  require  hospital  care  to  food  poisoning  at  a  local  restaurant  that  is 
mild  enough  to  not  warrant  an  emergency  room  visit.  Simulated  experiments  of 
alternative  systems  are  used  to  evaluate  alternate  systems  being  fielded. 

To  test  the  robustness  of  a  system,  sensitivity  analysis  is  performed.  There  are 
several  studies  showing  how  sensitivity  analysis  is  used  to  test  the  robustness  of  a 
healthcare  system  (Aktas,  Ulengin,  &  Sahin,  2007;  Hashimoto  &  Bell,  1996;  Dorr,  Horn, 
&  Smout,  2005;  Doubilet,  Begg,  Weinstein,  Braun,  &  McNeil,  1985;  Angus,  Kelley, 
Schmitz,  White,  &  Popovich,  2000).  One  such  study  is  performed  by  Aktas,  Ulengin, 
and  Sahin  (2007).  Aktas,  et  al.  performed  a  sensitivity  analysis  on  a  case  study  involving 
a  private  hospital  in  Turkey.  The  tomography  section  in  the  radiology  department  had  a 
problem  with  the  process  time  because  lengthy  time  spent  on  the  tomography  machines 
have  high  operating  costs;  additional  tomography  machines  also  have  a  high  cost  to 
purchase.  Also,  the  longer  a  doctor  spends  time  on  the  tomography  machines,  the  more 
dissatisfied  the  patients  will  be  due  to  patients  having  to  wait  in  long  queues.  In  order  to 
affect  change  in  this  system  to  reduce  patient  wait-times,  a  sensitivity  analysis  was 
conducted  to  identify  the  variables  that  are  the  most  sensitive  to  change  in  affecting  the 
process  time.  Aktas,  et  al.  found  that  the  process  time  is  very  sensitive  to  the  process 
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type;  if  different  process  types  are  offered  on  different  days,  then  it  can  solve  this  issue 
with  the  exception  of  one  type.  They  found  that  the  whole  abdomen  process  time  is 
exceptionally  long  by  itself.  Additionally,  Aktas,  et  al.  (2007)  found  that  the  process  time 
is  not  sensitive  to  technicians;  improving  the  technicians  will  have  no  effect,  if  any,  on 
the  process  time. 

While  Aktas,  et  al.  (2007)  used  sensitivity  analysis  to  look  at  reducing  the  process 
time.  Dorr,  Horn,  and  Smout  (2005)  took  a  different  approach  and  used  sensitivity 
analysis  to  evaluate  the  robustness  of  cost  estimates  ranging  from  hospitalization  costs  to 
registered  nurse  cost  per  hour.  Dorr,  et  al.  (2005)  found  that  hospitalization  costs  are 
most  sensitive  to  perturbations  from  nursing  home  residents. 

Hashimoto  and  Bell  (1996)  conducted  a  sensitivity  analysis  on  clinic  staffing  to 
evaluate  the  patient  time  in  the  clinic,  session  length,  and  idle  times  of  the  doctor  for  a 
single  clinic.  Hashimoto  and  Bell  (1996)  showed  that  patient  total  time  in  the  clinic  is 
sensitive  to  number  of  doctors  in  the  clinic;  increasing  the  doctor  staffing  level  decreased 
the  average  patient  time  in  system  by  18.3  minutes. 

Comparable  to  cost-value  analysis  and  cost-effectiveness  analysis,  cost-benefit 
analysis  studies  have  also  been  performed  in  the  healthcare  field  (Brown,  Brown, 

Sharma,  &  Landy,  2003;  Nord,  1993).  The  overarching  structure  of  these  analyses  stems 
from  costs  incurred  in  relation  to  gains  realized.  Brown,  Brown,  Sharma,  and  Landy 
(2003)  define  cost-benefit  analysis  as  a  measure  of  “both  the  costs  and  the  outcomes  of 
alternative  interventions  in  terms  of  dollars  (resources).” 

Several  studies  analyze  the  relationship  of  the  costs  to  gains  in  other  healthcare 
systems  (Nord,  1993;  Eichler,  Kong,  Gerth,  Mavros,  &  Jonsson,  2004;  van  den  Bemt,  et 
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al.,  2002).  Nord  (1993)  compared  the  costs  of  different  medical  interventions  to  the 
effect  treatments  have  on  patients.  This  resulted  in  giving  medical  decision  makers  the 
ability  to  prioritize  healthcare  programs  in  terms  of  cost  per  one  Saved  Young  Life 
Equivalent  (SAVE)  (Nord,  1993).  Eichler,  Kong,  Gerth,  Mavros,  and  Jonsson  (2004) 
took  a  different  approach  in  prioritizing  healthcare  resources  by  analyzing  the  cost  to  gain 
ratio  in  terms  of  an  acceptable  threshold  to  allocate  scarce  resources.  Van  den  Bemt, 
Postma,  van  Roon,  Chow,  Fijn,  and  Brouwers  (2002)  looked  into  reducing  prescription 
errors  by  hospital  pharmacy  staff  by  conducting  a  cost-benefit  analysis.  In  order  to  save 
money,  money  must  first  be  spent.  Van  den  Bemt  et  al.  (2002)  demonstrated  that  when 
the  hospital  invested  more  time  to  properly  prescribe  medication  to  patients,  a  net  cost  of 
€285  to  a  benefit  of  €9867  is  attributed  to  a  reduction  of  prescribing  errors. 

2.1  Research  Gap 

Despite  the  various  studies  conducted  on  sensitivity  and  cost-benefit  analysis,  no 
work  has  been  found  to  study  the  effects  of  increasing  patient  demand  have  on  a  system 
of  clinics.  In  particular,  no  study  is  found  that  analyzes  this  effect  due  to  (1)  increased 
military  deployments  or  (2)  a  surge  of  patient  “walk-ins”  due  to  a  mild  medical  incident 
in  the  local  area.  This  research  takes  a  novel  approach  of  using  both  sensitivity  analysis 
and  cost-benefit  analysis  to  evaluate  the  simulated  effects  of  an  increase  in  patient 
demand  on  a  system  of  clinics  where  some  resources  are  shared.  This  research  evaluates 
the  robustness  of  different  staffing  level  combinations  when  the  system  is  subjected  to 
deployment  demand  as  well  as  a  medical  incident  demand.  This  study  hypothesizes  that 
scenarios  with  increased  patient  demand  will  be  most  negatively  impacted  (longer  patient 
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wait-times)  for  alternative  scenario(s)  that  decrease  the  number  of  flight  medicine 


doctors. 

This  research  also  evaluates  the  savings  estimates  in  implementing  alternative 
scenarios:  where  the  staffing  levels  vary  in  the  system  of  clinics  while  maintaining  a 
zero-sum  manning  level  when  the  system  of  clinics  is  subjected  to  deployment  and 
medical  incident  demands.  This  study  hypothesizes  that  one  or  more  staffing  level  will 
have  a  statistically  lower  cost  to  implement  than  the  remaining  staffing  levels. 


3.  Methodology 

3.1  Baseline  Discrete-Event  Simulation  Model 

In  order  to  evaluate  the  increase  in  demand  on  a  system  of  clinics,  a  discrete-event 
simulation  (DES)  model  is  developed.  A  conceptual  model  of  the  system  of  clinics  is  the 
first  step  in  creating  the  model.  A  one-month  data  collection  effort  is  conducted  to 
collect  the  necessary  information  needed  to  develop  system  arrival  and  process  time 
probability  distributions  as  well  as  frequency  counts  for  the  model.  The  task  network  of 
the  model  is  developed  to  provide  a  blueprint  for  the  actual  model.  The  model  is  then 
developed  and  validated.  Once  the  model  is  validated,  it  is  ready  for  various  simulation 
experiments.  See  Chapter  3  for  a  detailed  discussion  of  this  process. 

3.2  Data  Collection 

After  the  DES  baseline  model  is  created  and  validated,  further  information  is 
needed  to  conduct  a  cost-benefit  analysis.  Cost  information  is  gathered  from  the  manager 
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of  the  88th  Aerospace  Medicine  Squadron,  Occupational  Medicine  Clinic.  The  manager 
provided  monthly  salary,  rounded  to  the  nearest  five  hundred  dollars,  for  each  of  the 
different  types  of  staff  members.  The  average  monthly  salary  is  calculated  for  each  staff 
member  type. 

3.3  Sensitivity  Analysis  Methodology 

In  order  to  evaluate  the  robustness  of  alternative  systems,  a  sensitivity  analysis  is 
conducted.  The  performance  measure  (average  total  wait-time)  is  analyzed  for  each 
alternative  scenario  as  well  as  the  baseline  scenario.  The  criteria  for  evaluating  the 
sensitivity  analysis  occurs  when  the  performance  measure  crosses  a  threshold  of  15 
minutes.  When  the  performance  measure  is  greater  than  15  minutes,  then  the  baseline  or 
alternative  system  is  considered  sensitive  to  the  respective  changing  environment.  The 
criteria  of  greater  than  15  minutes  stems  from  military  medical  facilities  using  15 
minutes  as  their  indicator  for  excessive  wait-time.  Additionally,  military  medical 
facilities  consider  15-minute  delays  as  a  “no-show”  for  patient  late  arrivals,  and  use  15 
minutes  as  the  criteria  for  patients  to  use  when  complaining  about  the  long  wait-time; 
other  medical  facilities  even  tell  patients  to  show  up  15  minutes  before  their  appointment 
(St.  Michael's  Emergency  Room  Commercials,  2015;  Peninsula  Children's  Clinic,  2015). 
The  results  are  shown  on  a  one-sided  tornado  diagram  in  Section  5.1  Sensitivity  Analysis 
Results  because  it  is  effective  in  rank  ordering  the  alternative  scenarios  from  the  most 
sensitive  at  the  top  to  the  least  sensitive  at  the  bottom.  The  tornado  diagram  is  one  sided 
because  it  is  intuitively  known  that  a  decrease  in  demand  would  not  affect  the  robustness 
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of  the  alternative  scenarios,  and  thus  decreases  in  demand  are  not  considered  in  this 
analysis. 

3.4  Cost-Benefit  Analysis  Methodology 

In  order  to  determine  the  alternative  scenario  with  the  lowest  cost  to  implement 
for  the  third  investigative  question,  a  cost-benefit  analysis  is  conducted.  The  lowest  cost 
to  implement  can  also  imply  the  highest  savings  incurred  upon  implementation.  A 
savings  estimate  is  used  as  the  criteria  to  determine  the  best  alternative  scenario  by  rank 
ordering  the  savings  estimates  from  highest  to  lowest  in  dollar  amounts.  The  savings 
estimate  is  in  terms  of  the  wait-time  reduced  or  gained  per  month  converted  to  a  dollar 
equivalent  plus  the  monthly  savings  incurred  of  removing  a  staff  member  from  the 
system  of  clinics: 

Savings  Estimate  =  Reduced  Wait-Time  Savings  +  Staff  Member  Reduction  Savings 

The  wait-time  reduction  or  gain  conversion  is  done  by  taking  the  wait-time  minutes, 
divide  it  by  60,  and  multiply  it  by  how  much  an  average  United  States  Air  Force  (USAF) 
employee  earns  per  hour;  according  to  careerbliss.com,  USAF  employees  earn  $23  per 
hour  on  average  (careerbliss.com,  2015).  The  savings  estimate  is  calculated  for  each  of 
the  three  patient  demand  environments:  baseline,  deployment,  and  medical  incident 
demand  environments.  Because  all  of  the  alternative  scenarios  add  an  additional 
optometrist  to  the  system  of  clinics,  see  Section  4.1  Experimental  Design,  the  cost 
associated  with  bringing  an  additional  optometrist  to  the  staff  is  removed  from  further 
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analysis  and  the  analysis  focuses  on  the  savings  aspect  of  removing  a  different  staff  type 
to  incur  a  savings.  If  an  alternative  scenario  isn’t  universally  ranked  first  in  all  three 
demand  scenarios,  then  to  decide  on  the  preferred  alternative  scenario  the  three  savings 
estimates  are  combined  into  a  roll-up  savings  estimate  where  all  three  patient  demands 
environments  are  given  equal  weight.  By  combining  the  three  savings  estimates  into  a 
combined  savings  estimate,  the  sensitivity  analysis  is  accounted  for  by  the  cost-benefit 
analysis. 


4.  Alternative  Systems 

To  evaluate  the  effect  of  deployment  demand  and  medical  incident  demand  has  on 
the  average  total  wait-time  of  the  system  of  clinics,  the  validated  baseline  simulation 
model  is  modified  to  incorporate  this  increase  as  well  as  incorporate  varying  staffing 
levels  from  each  clinic  in  the  system  of  clinics  to  determine  which  staff  position  change 
is  robust  against  increased  patient  demand.  The  staffing  levels  are  the  independent 
variables  while  the  patient  wait-time  is  the  dependent  variable  when  evaluating  the 
alternative  systems,  also  known  as  alternative  scenarios.  The  alternative  scenarios  are 
described  in  greater  detail  below  in  the  form  of  an  experimental  design. 

4.1  Experimental  Design 

Table  9  summarizes  the  simulation  experiments.  The  table  shows  the  staffing 
level  for  each  staff  type  as  well  as  the  system  of  clinics’  patient  demand  level  for  the 
baseline  scenario  (BS)  and  each  of  the  alternative  scenarios  (AS).  There  are  5  distinct 
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alternative  scenarios  similar  to  the  one  shown  in  Chapter  3,  except  this  experimental 
design  table  includes  the  additional  layer  of  patient  demand  level,  see  Chapter  3  for  a 
detailed  explanation  of  each  alternative  scenario  for  the  varying  staffing  level.  The 
patient  demand  level  for  the  baseline  scenario,  as  well  as  all  the  alternative  scenarios,  has 
a  200%  increase  of  patient  demand.  This  means  there  are  three  times  as  many  patients 
entering  the  system  of  clinics  than  the  baseline  demand.  The  total  number  of  additional 
patients  entering  the  system  is  the  same  for  both  the  deployment  demand  and  medical 
incident  demand  scenarios. 

Table  9:  Experimental  Design  Table 


STAFF  TYPE 

BS 

AS1 

AS2 

AS3 

AS4 

AS5 

Audiologist 

2  i 

2 

2 

2 

2 

Flight  Medicine  Doctor 

4 

4 

4 

4 

4 

Flight  Medicine  Nurse  or  Technician 

8 

8 

8 

8 

8 

Hearing  Conservation  Technician 

1 

1 

1 

1 

1 

1 

Occupational  Medicine  Doctor 

4 

4 

4 

4 

4 

Occupational  Medicine  Nurse  or  Technician 

4 

4 

4 

4 

4 

Optometrist 

1 

2 

2 

2 

2 

2 

Optometry  Nurse  or  Technician 

1 

1 

1 

1 

1 

1 

X  Ray  Technician 

1 

1 

1 

1 

1 

1 

PATIENT  DEMAND  LEVEL 

BS 

AS1 

AS2 

AS3 

AS4 

AS5  I 

System  of  Clinics 

4.2  Deployment  Demand 

In  order  to  account  for  deployment  demand,  the  model  is  slightly  modified.  The 
first  modification  is  a  change  in  the  arrival  rate.  To  take  into  account  the  200%  increase 
in  patient  demand  spread  throughout  the  day,  two  additional  create  nodes  of  the  same 
Weibull  distribution  used  in  Chapter  3  are  added  at  the  beginning  of  the  model  while 
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keeping  the  create  node  used  in  the  baseline  demand  scenario  intact;  all  three  create 
nodes  have  the  same  Weibull  distribution.  The  create  nodes  represent  the  interarrival 
rates  of  the  patients  entering  the  system  of  clinics. 

The  second  modification  is  a  change  in  probabilities  of  patients  visiting  a 
particular  clinic  as  well  as  a  change  in  probabilities  in  status  types  within  the  system  of 
clinics.  First,  we  establish  the  daily  frequency  count  of  how  many  patients  visit  each  of 
the  five  clinics  in  the  model  as  well  as  the  two  status  types  of  “scheduled  appointment” 
and  “walk-in”.  To  establish  the  daily  frequency  count  of  how  many  patients  visit  each  of 
the  five  clinics,  the  assigned  percentages  based  on  the  real-world  monthly  counts  are 
attributed  to  the  average  throughput  in  the  model  (Table  10).  Next,  the  increased 
throughput  from  the  patient  demand  increase  of  200%  (54  additional  patients)  is  added  to 
the  frequency  count  of  the  flight  medicine  clinic  since  the  flight  medicine  doctors  are  the 
only  ones  who  can  medically  clear  military  personnel  for  deployment.  The  54  additional 
patients  needing  medical  clearance  prior  to  deployment  have  appointment  status.  The 
new  frequency  counts  are  then  used  to  determine  the  probability  of  visiting  each  of  the 
clinics  and  status  of  the  patient.  The  new  percentages  for  each  clinic  as  well  as  new 
percentage  for  the  status  of  patients  are  integrated  into  the  model  decision  logic  (Table 
11).  Each  alternative  model  is  run  for  42  replications;  see  Chapter  3  for  the 
determination  of  the  required  number  of  replications. 
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Table  10:  Statistics  from  Baseline  Demand  to  Deployment  Demand  Part  1 


|  From  Data  Collection  | 

|  Clinic 

Count 

Percent 

Audiology 

68 

13.052% 

Flight  Medicine 

104 

19.962% 

Flearing  Conservation 

37 

7.102% 

Occupational  Medicine 

179 

34.357% 

Optometry 

133 

25.528% 

|  From  Data  Collection  | 

Status 

Count 

Percent 

Scheduled  Appointment 

474 

91% 

Walk  In 

45 

9% 

Simulation  System 
Number  Out 

Baseline 

(  Average 

27 

|  Expected  Avg.  Daily  Patients  (Sim_Baseline)  | 

Clinic 

Count 

Percent 

Audiology 

3.524 

13.052% 

Flight  Medicine 

5.390 

19.962% 

Flearing  Conservation 

1.917 

7.102% 

Occupational  Medicine 

9.276 

34.357% 

Optometry 

6.893 

25.528% 

|  Expected  Average  Daily  Patients  (Sim_Baseline)  | 

Status 

Count 

Percent 

Scheduled  Appointment 

24.659 

91% 

Walk  In 

2.341 

9% 

Table  11:  Statistics  from  Baseline  Demand  to  Deployment  Demand  Part  2 


Expected  Avg.  Daily  Patients  (Sim_Baseline) 

Clinic 

Count 

Percent 

Audiology 

3.524 

13.052% 

Flight  Medicine 

5.390 

19.962% 

Hearing  Conservation 

1.917 

7.102% 

Occupational  Medicine 

9.276 

34.357% 

Optometry 

6.893 

25.528% 

|  Expected  Average  Daily  Patients  (Sim_Baseline)  | 

Status 

Count 

Percent 

Scheduled  Appointment 

24.659 

91% 

Walk  In 

2.341 

9% 

Simulation  System 
Number  Out 

Patient 

Demand 

Level 

Demand  Increase 

200% 

Expected  Average 

81 

CAVEAT:  Only  Flight  Medicine  Clinic  is 
affected  by  DHA  Increase  in  Demand. 


Expected  Avg.  Daily  Patients  (Sim_200%  Inc.)  | 

Clinic 

Count 

Percent 

Audiology 

3.524 

4.351% 

Flight  Medicine 

59.390 

73.321% 

Hearing  Conservation 

1.917 

2.367% 

Occupational  Medicine 

9.276 

11.452% 

Optometry 

6.893 

8.509% 

Expected  Average  Daily  Patients  (Sim_SSl)  | 

Status 

Count 

Percent 

Scheduled  Appointment 

78.659 

97.110% 

Walk  In 

2.341 

2.890% 
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4.3  Medical  Incident  Demand 


The  medical  incident  demand  scenario  follows  the  same  experimental  design  table 
in  Table  9  with  the  exception  of  the  patient  demand  level  of  the  system  of  clinics;  it  will 
have  a  surge  of  54  patients  over  the  first  three  hours  of  operation.  To  do  this,  an 
additional  create  node  is  added  to  the  baseline  model  with  an  exponential  probability 
distribution  with  200  seconds  as  the  mean  parameter  that  stops  adding  patients  into  the 
system  after  3  hours  of  simulation  time  passes  for  each  replication.  The  exponential 
distribution  is  used  because  it  is  the  common  distribution  for  interarrival  times  (Kelton, 
Sadowski,  &  Zupick,  2015). 

The  second  modification  to  the  baseline  model  to  evaluate  the  effects  of  a  medical 
incident  demand  is  a  change  in  probabilities  of  patient  visiting  a  particular  clinic  as  well 
as  a  change  in  probabilities  in  status  types  within  the  system  of  clinics.  First,  we 
establish  the  daily  frequency  count  of  how  many  entities  visit  each  of  the  five  clinics  in 
the  model  as  well  as  the  two  status  types.  To  do  this,  the  assigned  percentages  based  on 
the  real-world  monthly  counts  are  attributed  to  the  average  throughput  in  the  model 
(Table  12).  Next,  the  increased  throughput  from  the  patient  demand  increase  of  54 
patients  in  the  surge  is  equally  split  and  added  to  the  frequency  count  of  the  flight  and 
occupational  medicine  clinics  since  they  are  both  equally  affected  by  this  surge.  Fifty- 
four  patients  are  also  added  to  the  walk-in  status  since  this  is  an  incident  that  occurred 
that  same  day  as  when  the  patient  enters  the  system  of  clinics.  The  new  frequency  counts 
are  then  used  to  determine  the  probability  of  visiting  each  of  the  clinics  and  status  of  the 
patient.  The  new  percentages  for  each  clinic  as  well  as  new  percentage  for  the  status  of 
patients  are  integrated  into  the  model  decision  logic  (Table  13).  Each  alternative  model 
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is  run  for  42  replications;  see  Chapter  3  for  the  determination  of  the  required  number  of 


replications. 


Table  12:  Statistics  from  Baseline  Demand  to  Medical  Incident  Demand  Part  1 


Simulation  System 
Number  Out 

Baseline 

|  Average 

27 

|  From  Data  Collection  | 

Clinic 

Count 

Percent 

Audiology 

68 

13% 

Flight  Medicine 

104 

20% 

Hearing  Conservation 

37 

7% 

Occupational  Medicine 

179 

34% 

Optometry 

133 

26% 

|  From  Data  Collection  | 

Status 

Count 

Percent 

Scheduled  Appointment 

474 

91% 

Walk  In 

45 

9% 

Expected  Average  Daily  Patients  (Sim_Baseline) 


|  Clinic 

Count 

Percent 

Audiology 

3.524 

13% 

Flight  Medicine 

5.390 

20% 

Hearing  Conservation 

1.917 

7% 

Occupational  Medicine 

9.276 

34% 

Optometry 

6.893 

26% 

|  Expected  Average  Daily  Patients  (Sim_Baseline)  | 

|  Status 

Count 

Percent 

Scheduled  Appointment 

24.659 

91% 

Walk  In 

2.341 

9% 

Table  13:  Statistics  from  Baseline  Demand  to  Medical  Incident  Demand  Part  2 


Simulation  System  Number  Out 

Surge 

Scenario 

Instant  Demand  Increase 

54 

Expected  Average 

81 

|  Expected  Average  Daily  Patients  (Sim_Baseline)  | 


|  Clinic 

Count 

Percent 

Audiology 

3.524 

13% 

Flight  Medicine 

5.390 

20% 

Hearing  Conservation 

1.917 

7% 

Occupational  Medicine 

9.276 

34% 

Optometry 

6.893 

26% 

|  Expected  Average  Daily  Patients  (Sim_Baseline)  | 

|  Status 

Count  Percent 

Scheduled  Appointment 

24.659  91% 

Walk  In 

2.341  9% 
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5.  Analysis  and  Results 

The  output  data  from  the  5  alternative  scenarios  for  the  deployment  demand  are 
presented  in  Table  14.  The  output  data  from  the  5  alternative  scenarios  for  the  medical 
incident  demand  are  presented  in  Table  15.  In  both  tables,  it  is  expected  to  see  an 
increase  in  average  total  wait-time  in  the  baseline  and  alternative  scenarios  for  the 
medical  incident  demand  because,  unlike  the  deployment  demand  where  54  additional 
patients  are  seen  throughout  the  day,  medical  incident  demand  are  seeing  a  surge  of  54 
additional  patients  within  the  first  3  hours  of  the  system  of  clinics  opening.  Sensitivity 
and  cost-benefit  analysis  are  conducted  and  once  the  analysis  is  complete,  the  results  of 
the  findings  are  discussed. 


Table  14:  Average  Wait-Time  of  the  Individual  Processes  and  System  of  Clinics  for  All 
Alternative  Scenarios  for  Deployment  Demand 


Process  Node  with  Queues:  Average  Wait-Time  in  Minutes  (Half-Width) 

BS 

AS1 

AS  2 

AS  3 

AS4 

AS  5 

2nd  Session  with  Flight  Medicine  Nurse  or  Tech 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

2nd  Session  with  Occupational  Medicine  Nurse  or  Tech 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

3rd  Session  with  Occupational  Medicine  Nurse  or  Tech 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

Check  In 

0.18(0.08) 

0.24(0.17) 

0.15(0.07) 

0.26(0.18) 

0.26(0.18) 

0.26(0.18) 

Check  Vitals_Flight  Medicine 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

Check  Vitals_Occupational  Medicine 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

Make  Follow  Up  Appointment 

0.29  (0.36) 

0.27  (0.3) 

0.18(0.25) 

0.21  (0.23) 

0.34  (0.36) 

0.34  (0.36) 

Perform  ECG_Flight  Medicine 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

Perform  ECG_Occupational  Medicine 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

Perform  Lab_Flight  Medicine 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

Perform  Lab_Occupational  Medicine 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

Perform  X  Ray 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

Prepare  to  See  Optometrist  with  Nurse  or  Tech 

1.54(0.6) 

1.12(0.66) 

1.52(0.78) 

1.63  (0.78) 

1.63  (0.79) 

1.63  (0.79) 

See  Flight  Medicine  Physician 

1.71  (0.28) 

1.9(0.48) 

14.66  (3.63) 

1.75  (0.36) 

1.91  (0.47) 

1.91  (0.47) 

See  Occupational  Medicine  Physician 

0.29(0.11) 

0.37(0.17) 

0.27(0.15) 

0.36(0.17) 

0.36(0.17) 

0.36(0.17) 

Visit  Audiologist 

0.46  (0.39) 

6.47  (2.62) 

0.35  (0.31) 

0.46  (0.37) 

0.46  (0.37) 

0.46  (0.37) 

Visit  Hearing  Conservation  Technician  (Hearing  Conservation  Clinic  Only) 

4.86(1.97) 

3.5  (1.53) 

4.13(1.64) 

3.44(1.36) 

3.57  (1.48) 

3.57  (1.48) 

Visit  Hearing  Conservation  Technician(Non  Hearing  Conservation  Clinics) 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

Visit  Optometrist 

15.72  (5.42) 

1.41  (0.79) 

2.21  (1.05) 

1.01  (0.68) 

0.95  (0.68) 

0.95  (0.68) 

Average  Total  Wait-Time  for  System  of  Clinics  in  Minutes  (Half-Width) 

20.93  (6.98) 

19.19(6.94) 

26.56(6.51) 

19.45  (6.28) 

20.43  (7.37) 

20.44  (7.37) 
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Table  15:  Average  Wait-Time  of  the  Individual  Processes  and  System  of  Clinics  for  All 


Alternative  Scenarios  for  Medical  Incident  Demand 


Process  Node  with  Queues:  Average  Wait-Time  in  Minutes  (Half-Width) 

BS 

AS1 

AS  2 

AS3 

AS  4 

AS  5 

2nd  Session  with  Flight  Medicine  Nurse  or  Tech 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

2nd  Session  with  Occupational  Medicine  Nurse  or  Tech 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

3rd  Session  with  Occupational  Medicine  Nurse  or  Tech 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

Check  In 

0.2(0.12) 

0.21(0.1) 

0.24(0.19) 

0.25(0.19) 

0.2(0.13) 

0.23(0.1) 

Check  Vitals_Flight  Medicine 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

Check  Vitals_Occupational  Medicine 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

Make  Follow' Up  Appointment 

0.21  (0.19) 

0.17(0.16) 

0.14(0.17) 

0.14(0.18) 

0.03  (0.02) 

0.2(0.16) 

Perform  ECGFlight  Medicine 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

Perform  ECG_Occupational  Medicine 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

Perform  Lab_Flight  Medicine 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

Perform  Lab_Occupational  Medicine 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

Perform  X  Ray 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

Prepare  to  See  Optometrist  with  Nurse  or  Tech 

5.18(1.71) 

3.78(1.14) 

4.51  (1.66) 

3.7  (1.26) 

4.57  (1.55) 

4.2(1.29) 

See  Flight  Medicine  Physician 

0.44(0.15) 

0.36  (0.09) 

1.52(0.62) 

0.49(0.13) 

0.42(0.11) 

0.47(0.12) 

See  Occupational  Medicine  Physician 

0.37(0.12) 

0.43(0.13) 

0.37(0.1) 

0.44(0.12) 

1.34(0.47) 

0.44(0.11) 

Visit  Audiologist 

1.5  (0.86) 

15.69(5.71) 

2.02  (0.93) 

1.67  (0.95) 

1.53  (0.91) 

1.61  (1.15) 

Visit  Hearing  Conservation  Technician  (Hearing  Conservation  Clinic  Only) 

5.94(1.6) 

6.31  (1.68) 

6.5  (1.5) 

7.24(1.6) 

6.75  (1.7) 

6.76(1.38) 

Visit  Hearing  Conservation  Technician(Non  Hearing  Conservation  Clinics) 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

0(0) 

Visit  Optometrist 

70.89  (15.79) 

10.01  (3.97) 

8.64  (2.92) 

6.66  (2.7) 

7.83  (3.35) 

8.87(3.18) 

Average  Total  Wait-Time  for  System  of  Clinics  in  Minutes  (Half-Width) 

57.2  (9.66) 

49.67  (9.65) 

48.91  (9.34) 

49.81  (9.54) 

49.17(9.71) 

52.38  (9.63) 

5.1  Sensitivity  Analysis  Results 

Based  on  the  criteria  of  greater  than  15  minutes  established  in  Section  3.3 
Sensitivity  Analysis  Methodology,  all  baseline  and  alternative  scenarios  indicate  that  the 
systems  considered  are  sensitive  to  deployment  demand,  see  Figure  7.  One  of  the 
alternative  systems  clearly  stood  out  from  the  other  alternative  systems  evaluated. 
Alternative  scenario  2  (AS2),  where  the  optometrist  staffing  level  is  increased  to  one  and 
the  flight  medicine  doctor  staffing  level  is  decreased  to  one,  shows  to  be  the  most 
sensitive  to  the  deployment  demand  when  compared  to  baseline  demand.  However,  the 
average  total  wait-time  is  not  statistically  significant  in  relation  to  the  average  total  wait¬ 
time  of  other  alternative  scenarios  at  a  95%  level  of  confidence,  see  Figure  8.  This  result 
failed  to  support  the  hypothesis  that  one  or  more  staffing  level  scenarios  will  have 
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statistically  lower  wait-times  than  other  staffing  levels  when  exposed  to  an  increase  in 


deployment  demand. 


One-Sided  Tornado  Diagram 

AS2:  Optometrist  +  1,  Flight  Medicine  Doctor  - 1 

BS:  Baseline 

AS5:  Optometrist  +  1,  Occupational  Medicine  Technician  - 1 

AS4:  Optometrist  +  1,  Occupational  Medicine  Doctor  - 1 

AS3:  Optometrist  +  1,  Flight  Medicine  Technician  - 1 

0  5  10  15  20  25  30 

AVERAGE  TOTAL  WAIT-TIME  (MINUTES) 

Deployment  Demand 

Figure  7:  One-Sided  Tornado  Diagram  of  Average  Total  Wait-Time  for  Deployment 

Demand 


IQ2.1  Deployment  Demand 

Average  Total  Wait-Time 
(XI 0  3) 


flipper  C.L. 
Mean 

Lower  C.L 


One-Way  ANOVA  Table  :  IQ2.1  Deployment  Demand 

LI  -  AvgTotalWaitTime  Baseline  200%  Increase 
L2  -  AvgTotalWaitTime  AS1  200%  Increase 
L3  -  AvgTotalWaitTime  AS2  200%  Increase 
L4  -  AvgTotalWaitTime  AS3  200%  Increase 
L5  -  AvgTotalWaitTime  AS 4  200%  Increase 
L6  -  AvgTotalWaitTime  AS5  200%  Increase 


SOURCE  OF  VARIATION 

SUM  SQUARES 

DF 

MEAN  SQUARES 

F-EXP 

BETWEEN  TREATMENTS 
ERROR  (W. TREATMENTS) 

5.599e+006 

4 . 365e+00B 

5 

246 

1. 12e+006 

1.77e+006 

0.631 

TOTAL 

FAIL  TO  REJECT  HO 

4.421e+008  251 

=>  MEANS  ARE  EQUAL  AT 

F-CRIT 

'  0.05  LEVEL 

2.251 

Figure  8:  Statistical  Analysis:  One-Way  ANOVA  of  Baseline  and  Alternative  Scenarios 

Subjected  to  Deployment  Demand 
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Based  on  the  criteria  of  greater  than  15  minutes  established  in  Section  3.3 
Sensitivity  Analysis  Methodology,  all  baseline  and  alternative  scenarios  indicate  that  the 
systems  considered  are  sensitive  to  medical  incident  demand,  see  Figure  9;  but,  like  the 
deployment  demand  analysis,  the  average  total  wait-time  of  these  alternatives  are  not 
statistically  significant  in  relation  to  each  other  at  a  95%  level  of  confidence,  see  Figure 
10.  Figure  10  fails  to  support  the  hypothesis  that  one  or  more  staffing  level  scenarios  will 
have  statistically  lower  wait-times  than  other  staffing  levels  when  exposed  to  a  medical 
incident  demand. 


One-Sided  Tornado  Diagram 

BS:  Baseline 

AS5:  Optometrist  +  1,  Occupational  Medicine  Technician  - 1 
AS3:  Optometrist  + 1,  Flight  Medicine  Technician  - 1 
AS1:  Optometrist  +1,  Audiologist  - 1 
AS4:  Optometrist  +  1,  Occupational  Medicine  Doctor  - 1 

0  10  20  30  40  50  60  70 

AVERAGE  TOTAL  WAIT-TIME  (MINUTES) 


Medical  Incident  Demand 


Figure  9:  One-Sided  Tornado  Diagram  of  Average  Total  Wait-Time  for  Medical  Incident 

Demand 
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IQ2.2  Medical  Incident  Demand 


Average  Total  Wait-Time 
(XI 0  3) 


|  Upper  C.L. 
Mean 
I  Lower  C.L. 


1  I  I  I  I  I 


L2  L3  L4  L5 

Baseline  and  Alternative  Scenarios 


One-Way  ANOVA  Table  :  IQ2.2  Medical  Incident  Demand 


LI  -  AvgTotalWaitTime  Baseline  54  Patient  Surge  First  3Hrs 
L2  -  AvgTotalWaitTime  AS1  54  Patient  Surge  First  3Hrs 

L3  -  AvgTotalWaitTime  AS2  54  Patient  Surge  First  3Hrs 

L4  -  AvgTotalWaitTime  AS3  54  Patient  Surge  First  3Hrs 

L5  -  AvgTotalWaitTime  AS4  54  Patient  Surge  First  3Hrs 

L6  -  AvgTotalWaitTime  AS5  54  Patient  Surge  First  3Hrs 


SOURCE  OF  VARIATION  SUM  SQUARES 


DF  MEAN  SQUARES 


BETWEEN  TREATMENTS  7.726e+006 

ERROR  (W. TREATMENTS)  8.383e+008 


5 

246 


1. 545e+006 
3.41e+006 


TOTAL  8.46e+008  251  F-CRIT 

FAIL  TO  REJECT  HO  =>  MEANS  ARE  EQUAL  AT  0.05  LEVEL 


Figure  10:  Statistical  Analysis:  One-Way  ANOVA  of  Baseline  and  Alternative  Scenarios 

Subjected  to  Medical  Incident  Demand 


5.2  Cost-Benefit  Analysis  Results 

This  section  provides  the  cost-benefit  analysis  results  under  the  three  demand 
environments  already  stated:  baseline  demand,  deployment  demand,  and  medical 
incident  demand.  The  best  alternatives  in  terms  of  staff  reduction  dollar  savings  and 
wait-time  reduction  dollar  equivalent  are  presented. 

For  a  cost-benefit  analysis  on  baseline  demand  and  based  on  the  criteria  of  rank 
ordering  the  savings  estimate  from  the  highest  to  the  lowest  in  terms  of  dollar  amounts 
established  in  Section  3.4  Cost-Benefit  Analysis  Methodology,  alternative  scenario  4  is 
the  highest  ranked  alternative  when  compared  to  the  other  alternative  scenarios.  The 
reason  why  increasing  the  optometrist  staffing  level  by  one  and  decreasing  the 
occupational  medicine  doctor  staffing  level  by  one  is  the  best  alternative  in  terms  of  the 
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highest  savings  incurred  is  because  it  saves  the  system  of  clinics  a  monthly  average  of 
$15,800,  rounded  to  the  nearest  $100,  for  removing  an  occupational  medicine  doctor,  see 
Table  16.  Alternative  system  4  is  the  best  solution  to  implement  when  the  system  of 
clinics  is  subject  to  baseline  demand.  A  one-way  ANOVA  is  used  to  test  the  significance 
of  the  mean  savings  estimate  of  any  of  the  alternative  scenarios.  This  study  finds  that  at 
least  one  alternative  scenario  has  a  statistically  lower  cost  to  implement  than  the 
remaining  staffing  levels  in  a  baseline  demand  environment;  p-value  =  0.000,  see  Figure 
11.  In  order  to  determine  if  the  difference  in  the  dependent  variable  (savings  estimate) 
between  the  alternative  scenarios  is  significant,  a  post-hoc  statistical  analysis  (Tukey 
Pairwise  Comparison)  is  conducted.  Figure  12  and  Figure  13  both  indicate  that  AS4 
paired  with  each  of  the  other  alternative  scenarios  all  have  a  significant  difference  in  the 
dependent  variable.  When  AS4  is  compared  with  other  alternative  scenarios,  it  doesn’t 
contain  the  value  zero  in  Figure  12,  and  AS4  is  not  grouped  with  any  other  alternative 
scenarios  in  Figure  13.  If  the  medical  decision  maker  is  only  looking  at  the  analysis  from 
purely  an  average  wait-time  reduction  perspective,  then  AS2  and  AS3  would  both  tie  as 
the  best  solutions,  see  Table  16.  Unfortunately,  it  costs  money  to  affect  a  reduction  in 
wait-time  by  changing  staffing  levels  in  the  system  of  clinics. 


61 


Table  16:  Cost-Benefit  Analysis  Table  -  Baseline  Demand 


Baseline  Demand 

Alternative 

Scenario 

Staff  Type 

Staff  Type 
Salary  Savings 
(Monthly) 

Average 

Wait-Time 

Reduced  per  Month 
in  Minutes  (Half-Width,  95%) 

Wait-Time  Reduced 
Dollar  Equivalent 
(60  Minutes  =  $23.00) 

Wait-Time  Reduced 

Dollar  Equivalent  + 
Staff  Savi  ngs 
(Half-Width,  95%) 
Rounded  near  $100 

Rank 

ASl 

Audiologist  - 1 

$8,000.00 

2,946  (404) 

$1,129 

$9,100  ($200) 

3 

AS2 

Fit.  Med.  Doctor  -1 

$9,000.00 

3,455  (302) 

$1,324 

$10,300  ($100) 

2 

AS3 

Fit.  Med.  Nurse/Tech  -1 

$4,666.67 

3,455  (302) 

$1,324 

$6,000  ($100) 

5 

AS4 

Occ.  Med.  Doctor -1 

$14,500.00 

3,391(311) 

$1,300 

$15,800  ($100) 

i 

AS5 

Occ.  Med.  Nurse/Tech  -1 

$5,750.00 

3,451  (304) 

$1,323 

$7,100  ($100) 

4 

One-way  ANOVA:  AS1,  AS2,  AS3,  AS4,  AS5 

Method 

Null  hypothesis  All  means  are  equal 

Alternative  hypothesis  At  least  one  mean  is  different 
Significance  level  a  =  0.05 

Equal  variances  were  assumed  for  the  analysis. 


Factor  Information 

Factor  Levels  Values 

Factor  5  AS1,  AS2,  AS3,  AS4,  ASS 


Analysis  of  Variance 

Source  DF  Adj  SS  Adj  MS  F-Value  P-Value 
Factor  4  2460122088  615030522  3805.33  0.000 

Error  205  33132838  161624 

Total  209  2493254927 


Model  Summary 

S  R-sq  R-sq(adj)  R-sq(pred) 
402.024  98.67%  98.651  98.61% 


Means 


Factor  N  Mean  StDev 

AS1  42  9129.5  496.7 

AS2  42  10324.5  371.1 

AS3  42  5991.1  371.1 

AS4  42  15799.8  382.4 

ASS  42  7072.7  373.8 


95%  Cl 

(  9007.2,  9251.8) 
(10202.2,  10446.8) 
(  5868.8,  6113.5) 
(15677.5,  15922.1) 
(  6950.4,  7195.0) 


Pooled  StDev  =  402.024 


Figure  11:  One-way  ANOVA  for  AS1  through  AS5  (Baseline  Demand) 
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Tukey  Simultaneous  95%  CIs 

Difference  of  Means  for  AS1 ,  AS2,  AS3,  AS4,  AS5  (Baseline  Demand) 


AS2  AS1 

!  W 

AS3  -  AS1 

w  ! 

AS4  AS1 

!  M 

AS5  -  AS1 

1*  ! 

AS3  -  AS2 

i+i  ! 

i 

AS4-  AS2 

i 

■  w 

i 

AS5  -  AS2 

i 

w  j 

AS4  -  AS3 

AS5  -  AS3 

j 

AS5  -  AS4 

1+  j 

-10000  -5000  0  5000  10000 

If  an  interval  does  not  contain  zero,  the  corresponding  means  are  significantly  different. 


Figure  12:  Tukey  Simultaneous  95%  Confidence  Intervals  (Baseline  Demand) 


Tukey  Pairwise  Comparisons 

Grouping  Information  Using  the  Tukey  Method  and  95%  Confidence 


Factor 

N 

Mean  Grouping 

AS4 

42 

15799.8  A 

AS2 

42 

10324.5  B 

AS1 

42 

9129.5  C 

AS5 

42 

7072.7  D 

AS3 

42 

5991.1  E 

Means  that  do  not  share  a  letter  are  significantly  different. 


Figure  13:  Tukey  Pairwise  Comparisons  (Baseline  Demand) 


For  a  cost-benefit  analysis  on  deployment  demand  and  based  on  the  criteria  of 


rank  ordering  the  savings  estimate  from  the  highest  to  the  lowest  in  terms  of  dollar 


amounts  established  in  Section  3.4  Cost-Benefit  Analysis  Methodology,  alternative 


scenario  4  is  the  highest  ranked  alternative  when  compared  to  the  other  alternative 
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scenarios.  The  reason  why  increasing  the  optometrist  staffing  level  by  one  and 
decreasing  the  occupational  medicine  doctor  staffing  level  by  one  is  the  best  alternative 
in  terms  of  the  highest  savings  incurred  is  because  it  saves  the  system  of  clinics  a 
monthly  average  $14,600,  rounded  to  the  nearest  $100,  for  removing  an  occupational 
medicine  doctor,  see  Table  17.  Alternative  system  4  is  the  best  solution  to  implement 
when  the  system  of  clinics  is  subject  to  deployment  demand.  A  one-way  ANOVA  is 
used  to  test  the  significance  of  the  mean  savings  estimate  of  any  of  the  alternative 
scenarios.  It  is  found  that  at  least  one  alternative  scenario  is  found  to  have  a  statistically 
lower  cost  to  implement  than  the  remaining  staffing  levels  in  a  deployment  demand 
environment;  p-value  =  0.000,  see  Figure  14.  In  order  to  determine  if  the  difference  in 
the  dependent  variable  (savings  estimate)  between  the  alternative  scenarios  is  significant, 
a  post-hoc  statistical  analysis  (Tukey  Pairwise  Comparison)  is  conducted.  Figure  15  and 
Figure  16  both  indicate  that  AS4  paired  with  each  of  the  other  alternative  scenarios  all 
have  a  significant  difference  in  the  dependent  variable.  When  AS4  is  compared  with 
other  alternative  scenarios,  it  doesn’t  contain  the  value  zero  in  Figure  15,  and  AS4  is  not 
grouped  with  any  other  alternative  scenarios  in  Figure  16.  If  the  medical  decision  maker 
is  only  looking  at  the  analysis  from  purely  an  average  wait-time  reduction  perspective, 
then  AS1  is  the  best  solution,  see  Table  17.  Unfortunately,  it  costs  money  to  affect  a 
reduction  in  wait-time  by  changing  staffing  levels  in  the  system  of  clinics. 
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Table  17:  Cost-Benefit  Analysis  Table  -Deployment  Demand 


Deployment  Demand 

Alternative 

Scenario 

Staff  Type 

Staff  Type 
Salary  Savings 
(Monthly) 

Average 

Wait-Time 

Reduced  per  Month 
in  Minutes  (Half-Width,  95%) 

Wait-Time  Reduced 
Dollar  Equivalent 
(60  Minutes  =  $23.00) 

Wait-Time  Reduced 

Dollar  Equivalent  + 
Staff  Savi  ngs 
(Half-Width,  95%) 
Rounded  near  $100 

Rank 

ASl 

Audiologist  - 1 

$8,000.00 

988(3,933) 

$379 

$8,400  ($1,500) 

2 

AS2 

Fit.  Med.  Doctor  -1 

$9,000.00 

-3,191(3,688) 

-$1,223 

$7,800  ($1,400) 

3 

AS3 

Fit.  Med.  Nurse/Tech  -1 

$4,666.67 

837(3,561) 

$321 

$5,000  ($1,400) 

5 

AS4 

Occ.  Med.  Doctor -1 

$14,500.00 

281(4,177) 

$108 

$14,600  ($1,600) 

i 

AS5 

Occ.  Med.  Nurse/Tech  -1 

$5,750.00 

280(4,177) 

$107 

$5,900  ($1,600) 

4 

One-way  ANOVA:  AS1,  AS 2,  AS 3,  AS4,  AS5 

Method 

Null  hypothesis  All  means  are  equal 

Alternative  hypothesis  At  least  one  mean  is  different 
Significance  level  a  =  0.05 

Equal  variances  were  assumed  for  the  analysis. 


Factor  Information 

I 

Factor  Levels  Values 

Factor  5  AS1,  AS2,  AS3,  AS4,  AS5 


Analysis  of  Variance 

Source  DF  Adj  SS  Adj  MS  F-Value  P-Value 
Factor  4  2394204603  59e551151  25.80  0.000 

Error  205  4755277135  23196474 

Total  209  7149481739 


Model  Summary 

S  R-sq  R-sq(adj)  R-sq(pred) 
4816.27  33.491  32.191  30.20% 


Means 

Factor  N  Mean  StDev  95%  Cl 

AS1  42  8379  4838  (  6913,  9844) 

AS2  42  7777  4537  (  6312,  9242) 

AS3  42  4987  4380  (  3522,  6453) 

AS4  42  14608  5139  (13142,  16073) 

ASS  42  5857  5138  (  4392,  7323) 

Pooled  StDev  =  4816.27 


Figure  14:  One-way  ANOVA  for  AS1  through  AS5  (Deployment  Demand) 
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Tukey  Simultaneous  95%  CIs 

Difference  of  Means  for  AS1,  AS2,  AS3,  AS4,  AS5  (Deployment  Demand) 

AS2  AS1 
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i 
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If  an  interval  does  not  contain  zero,  the  corresponding  means  are  significantly  different. 

Figure  15:  Tukey  Simultaneous  95%  Confidence  Intervals  (Deployment  Demand) 


Tukey  Pairwise  Comparisons 

Grouping  Information  Using  the  Tukey  Method  and  95%  Confidence 


Factor 

N 

Mean 

Grouping 

AS4 

42 

14608 

A 

AS1 

42 

8379 

B 

AS2 

42 

7777 

B  C 

AS5 

42 

5857 

B  C 

AS3 

42 

4987 

C 

Means  that  do  not  share  a  letter  are  significantly  different. 


Figure  16:  Tukey  Pairwise  Comparisons  (Deployment  Demand) 


For  a  cost-benefit  analysis  on  medical  incident  demand  and  based  on  the  criteria 


of  rank  ordering  the  savings  estimate  from  the  highest  to  the  lowest  in  terms  of  dollar 


amounts  established  in  Section  3.4  Cost-Benefit  Analysis  Methodology,  alternative 


scenario  4  is  the  highest  ranked  alternative  when  compared  to  the  other  alternative 
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scenarios.  The  reason  why  increasing  the  optometrist  staffing  level  by  one  and 
decreasing  the  occupational  medicine  doctor  staffing  level  by  one  is  the  best  alternative 
in  terms  of  the  highest  savings  incurred  is  because  it  saves  the  system  of  clinics  a 
monthly  average  $16,200,  rounded  to  the  nearest  $100,  for  removing  an  occupational 
medicine  doctor,  see  Table  18.  Alternative  system  4  is  the  best  solution  to  implement 
when  the  system  of  clinics  is  subject  to  medical  incident  demand.  A  one-way  ANOVA  is 
used  to  test  the  significance  of  any  of  the  alternative  scenarios.  This  study  finds  that  at 
least  one  alternative  scenario  has  a  statistically  lower  cost  to  implement  than  the 
remaining  staffing  levels  in  a  baseline  demand  environment;  p-value  =  0.000,  see  Figure 
17.  In  order  to  determine  if  the  difference  in  the  dependent  variable  (savings  estimate) 
between  the  alternative  scenarios  is  significant,  a  post-hoc  statistical  analysis  (Tukey 
Pairwise  Comparison)  is  conducted.  Figure  18  and  Figure  19  both  indicate  that  AS4 
paired  with  each  of  the  other  alternative  scenarios  all  have  a  significant  difference  in  the 
dependent  variable.  When  AS4  is  compared  with  other  alternative  scenarios,  it  doesn’t 
contain  the  value  zero  in  Figure  18,  and  AS4  is  not  grouped  with  any  other  alternative 
scenarios  in  Figure  19.  If  the  medical  decision  maker  is  only  looking  at  the  analysis  from 
purely  a  wait-time  reduction  perspective,  then  AS2  is  the  best  solution,  see  Table  18. 
Unfortunately,  it  costs  money  to  affect  a  reduction  in  wait-time  by  adding  an  optometrist 
to  the  system  of  clinics.  Because  all  three  cost-benefit  analysis  tables  for  the  baseline, 
deployment,  and  medical  incident  demands  indicate  that  AS4  is  ranked  first,  a  combined 
table  for  all  three  patient  demand  environments  is  not  necessary. 
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Table  18:  Cost-Benefit  Analysis  Table  -  Medical  Incident  Demand 


Medical  Incident  Demand 

Alternative 

Scenario 

Staff  Type 

Staff  Type 
Salary  Savings 
(Monthly) 

Average 

Wait-Time 

Reduced  per  Month 
in  Minutes  (Half-Width,  95%) 

Wait-Time  Reduced 
Dollar  Equivalent 
(60  Minutes  =  $23.00) 

Wait-Time  Reduced 

Dollar  Equivalent  + 
Staff  Savings 
(Half-Width,  95%) 
Rounded  near  $100 

Rank 

ASl 

Audiologist  - 1 

$8,000.00 

4,272(5,468) 

$1,638 

$9,600  ($2,100) 

3 

AS2 

Fit.  Med.  Doctor  -1 

$9,000.00 

4,705(5,296) 

$1,804 

$10,800  ($2,000) 

2 

AS3 

Fit.  Med.  Nurse/Tech  -1 

$4,666.67 

4,192(5,408) 

$1,607 

$6,300  ($2,100) 

5 

AS4 

Occ.  Med.  Doctor -1 

$14,500.00 

4,555(5,506) 

$1,746 

$16,200  ($2,100) 

i 

AS5 

Occ.  Med.  Nurse/Tech  -1 

$5,750.00 

2,735(5,460) 

$1,048 

$6,800  ($2,100) 

4 

One-way  ANOVA:  AS1,  AS2,  AS3,  AS4,  AS5 

Method 

Null  hypothesis  All  means  are  equal 

Alternative  hypothesis  At  least  one  mean  is  different 
Significance  level  a  =  0.05 

Equal  variances  were  assumed  for  the  analysis. 


Factor  Information 

Factor  Levels  Values 

Factor  5  AS1,  AS2,  AS3,  AS4,  AS5 


Analysis  of  Variance 

Source  DF  Adj  SS  Adj  MS  F-Value  P-Value 

Factor  4  2684429268  671107317  15.05  0.000 

Error  205  9139715624  44583979 

Total  209  11824144892 


Model  Summary 

S  R-sq  R-sq(adj)  R-sq(pred) 
6677.12  22.70%  21.19%  18.89% 


Means 

Factor  N  Mean  StDev  95%  Cl 

AS1  42  9638  6726  (  7606,  11669) 

AS2  42  10804  6515  (  e772,  12e35) 

AS3  42  6274  6652  (  4242,  8305) 

AS4  42  16246  6773  (14215,  18277) 

AS 5  42  6798  6717  (  4767,  8830) 

Pooled  StDev  =  6677.12 


Figure  17:  One-way  ANOVA  for  AS1  through  AS5  (Medical  Incident  Demand) 
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Tukey  Simultaneous  95%  CIs 

Difference  of  Means  for  AS1,  AS2,  AS3,  AS4,  AS5  (Medical  Incident  Demand) 
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If  an  interval  does  not  contain  zero,  the  corresponding  means  are  significantly  different. 

Figure  18:  Tukey  Simultaneous  95%  Confidence  Intervals  (Medical  Incident  Demand) 


[fukey  Pairwise  Comparisons 

Grouping  Information  Using  the  Tukey  Method  and  95%  Confidence 


Factor 

N 

Mean 

Grouping 

AS  4 

42 

16246 

A 

AS  2 

42 

10804 

B 

AS1 

42 

9638 

B  C 

AS  5 

42 

6798 

B  C 

AS  3 

42 

6274 

C 

Means  that  do  not  share  a  letter  are  significantly  different. 


Figure  19:  Tukey  Pairwise  Comparisons  (Medical  Incident  Demand) 
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6.  Conclusion 


6.1  Sensitivity  Analysis  Conclusion 

There  are  insights  gained  from  the  sensitivity  and  cost-benefit  analyses 
conducted.  It  is  shown  that  the  average  total  wait-times  of  the  baseline  and  alternative 
systems  are  sensitive  to  a  deployment  demand  but  are  not  statistically  significant  in 
relation  to  all  the  baseline  and  alternative  scenarios;  lack  of  statistical  significance  among 
the  baseline  and  alternative  scenarios  indicate  that  the  average  total  wait-time  of  one 
scenario  is  not  any  more  sensitive  than  the  average  total  wait-time  of  the  rest  of  the 
scenarios.  If  the  increase  in  demand  is  greater  than  200%,  then  it  is  expected  to  see 
greater  differences  in  the  average  total  wait-time,  especially  if  the  system  of  clinics 
reduce  the  staffing  levels  of  the  flight  medicine  doctor  even  though  the  difference  in 
average  total  wait-time  is  not  considered  statistically  significant  among  the  alternative 
scenarios  at  200%.  Reducing  the  staffing  levels  of  the  flight  medicine  doctor  at 
deployment  demand  levels  greater  than  200%  may  make  the  difference  in  average  total 
wait-time  significant  among  the  alternative  scenarios.  It  is  also  shown  that  the  average 
total  wait-times  of  the  baseline  and  alternative  systems  are  sensitive  to  a  medical  incident 
demand  but  are  not  statistically  significant  in  relation  to  all  the  baseline  and  alternative 
scenarios  in  the  same  medical  incident  demand  environment;  lack  of  statistical 
significance  among  the  difference  of  the  average  total  wait-time  of  the  scenarios  indicate 
that  the  average  total  wait-time  of  one  scenario  is  not  any  more  sensitive  than  the  average 
total  wait-time  of  the  rest  of  the  scenarios.  If  the  number  of  additional  patients  are 
increased,  the  time  length  of  the  surge  of  additional  patients  coming  in  to  the  system  of 
clinics  is  reduced  (less  than  3  hours),  or  a  combination  of  both,  then  it  is  expected  to  see 
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greater  differences  in  the  average  total  wait-time,  regardless  of  which  scenario  is 
implemented. 

6.2  Cost-Benefit  Analysis  Conclusion 

The  results  from  the  cost-benefit  analyses  clearly  show  that  implementing 
alternative  scenario  4  would  yield  the  highest  return-on-investment  for  the  lowest  cost  to 
implement  in  all  three  patient  demand  scenarios;  the  difference  in  mean  savings  estimate 
of  alternative  scenario  4  compared  to  each  of  the  other  alternative  scenarios  is  statistically 
significant  in  baseline,  deployment,  and  medical  incident  demand  environments.  This  is 
primarily  due  to  the  high  cost  of  keeping  an  occupational  medicine  doctor  on  the  payroll. 

6.3  Future  Work 

Future  work  includes  evaluating  other  alternative  scenarios  beyond  the  five 
studied  and  subjecting  these  to  an  increase  in  patient  demand.  For  example,  reducing 
wait-time  could  be  achieved  if  both  the  optometrist  and  flight  medicine  doctor  increased 
staffing  levels  while  reducing  some  of  the  nurse/technicians  in  an  increased  patient  and 
environment.  Additional  future  work  includes  looking  into  different  patient  demand 
environments.  It  would  be  interesting  to  determine  what  the  deployment  and  medical 
incident  demand  levels  are  at  the  robustness  threshold  as  well  as  evaluate  other  demand 
levels.  The  same  analyses  conduced  in  this  research  should  be  performed  at  other 
military  installations  to  determine  if  their  processes  can  be  improved  as  well. 
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V.  Conclusions  and  Recommendations 


Chapter  Overview 

The  purpose  of  this  chapter  is  to  answer  the  investigative  questions,  provide 
insights,  provide  a  solution  recommendation,  and  provide  recommendations  for  future 
research.  Discrete-event  simulation,  sensitivity  analysis,  and  cost-benefits  analysis  are 
utilized  in  this  research  to  determine  if  different  staffing  levels  can  affect  the  patient’s 
overall  wait-time;  which  of  the  different  staffing  levels  is  the  most  robust  as  patient 
demand  increases;  and  which  of  the  different  staffing  levels  has  the  lowest  cost  to 
implement.  The  results  of  this  research  provide  insights  into  the  current  military 
healthcare  system  that  is  comparable  to  the  system  of  clinics  and  can  be  used  to  improve 
the  process  of  the  current  system  of  clinics  at  Wright-Patterson  AFB,  OH. 


Investigative  Question  1:  How  can  staffing  levels  be  adjusted  to  improve  the 
patient’s  overall  wait-time  in  the  system  of  clinics? 

The  results  of  the  simulation  experiments  indicate  that  adding  an  additional 
optometrist  can  substantially  reduce  the  average  wait-time  of  the  optometry  clinic  and 
thus  the  average  time  in  system  for  the  system  of  clinics.  Adding  one  more  optometrist 
can  reduce  the  patients  visiting  the  optometry  clinic  in  a  system  of  clinics  by  as  much  as 
16.6  minutes,  on  average.  In  a  military  setting,  that  is  16.6  minutes  of  time  not  wasted 
waiting  to  see  the  optometrist  and  using  that  gained  time  back  in  performing  official 
military  duties.  If  adding  one  more  optometrist  is  not  feasible  when  implementing  the 
alternative  system,  then  load  sharing  with  other  optometrists  at  the  main  hospital  at 
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WPAFB  could  be  a  viable  alternative.  Evaluating  the  load  sharing  scenario  is  left  to 
future  work. 

Due  to  the  difficulties  in  increasing  military  unit  manpower  requirement  this 
study  investigated  alternatives  that  achieved  zero-sum  manning  for  the  system  of  clinics. 
Simulation  of  five  staffing  changes  reveals  that  there  are  4  viable  options  for  significantly 
reducing  total  average  wait-time  in  the  system  with  insignificant  impacts  to  the  individual 
process  average  wait-times.  These  options  are  reducing  a  flight  medicine  doctor,  flight 
medicine  nurse/technician,  occupational  medicine  doctor,  or  occupational  medicine 
nurse/technician. 


Investigative  Question  2:  Which  staffing  level  solution  is  the  most  robust  as  patient 
demand  increases? 

It  is  shown  that  the  average  total  wait-times  of  the  baseline  and  alternative 
systems  are  sensitive  to  deployment  demand  but  are  not  statistically  significant  in  relation 
to  the  difference  of  average  total  wait-time  of  all  the  scenarios;  all  systems  have  an 
average  total  wait-time  of  15  minutes  or  more.  However,  lack  of  statistical  significance 
in  the  difference  of  the  average  total  wait-time  among  the  scenarios  in  a  deployment 
demand  environment  indicate  that  the  average  total  wait-time  of  one  scenario  is  not  any 
more  sensitive  than  the  average  total  wait-time  of  the  rest  of  the  scenarios.  If  the  increase 
in  deployment  demand  is  greater  than  200%,  then  it  is  expected  to  see  greater  increase  in 
the  average  total  wait-time,  especially  if  you  reduce  the  staffing  levels  of  the  flight 
medicine  doctor.  This  study  also  finds  that  the  average  total  wait-times  of  the  baseline 
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and  alternative  systems  are  sensitive  to  a  medical  incident  demand  but  are  not  statistically 
significant  in  relation  to  the  difference  of  average  total  wait-time  in  all  the  scenarios;  lack 
of  statistical  significance  of  the  difference  of  average  total  wait-time  among  the  scenarios 
in  a  medical  incident  demand  environment  indicate  that  the  average  total  wait-time  of  one 
scenario  is  not  any  more  sensitive  than  the  average  total  wait-time  of  the  rest  of  the 
scenarios.  If  the  number  of  additional  patients  are  increased,  the  time  length  of  the  surge 
of  additional  patients  coming  in  to  the  system  of  clinics  is  reduced  (less  than  3  hours),  or 
a  combination  of  both,  then  it  is  expected  to  see  greater  increase  in  the  average  total  wait¬ 
time,  regardless  of  which  scenario  is  implemented.  It  is  found  that  none  of  the  scenarios 
are  robust  against  the  demands  of  deployments  and  medical  incidents  based  on  the 
sensitivity  criteria  established  in  Chapter  4. 


Investigative  Question  3:  Which  system  improvement  solution  has  the  lowest  cost  to 
implement? 

The  results  from  the  cost-benefit  analyses  clearly  show  that  implementing 
alternative  scenario  4  would  yield  the  lowest  cost  to  implement  due  to  a  high  savings 
incurred  if  implemented  in  all  three  patient  demand  scenarios;  the  difference  of  the 
average  savings  estimate  between  alternative  scenario  4  and  each  of  the  other  alternative 
scenarios  is  statistically  significant  in  baseline,  deployment,  and  medical  incident  demand 
environments.  This  is  due  to  the  high  cost  of  keeping  an  occupational  medicine  doctor  on 
the  payroll. 
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Research  Question:  How  can  the  total  wait-time  patients  experience  in  the  military 
system  of  clinics  be  cost-effectively  reduced  during  baseline  demand  and  when 
patient  demand  increases  as  the  clinics  within  the  system  of  clinics  compete  for 
scarce  resources? 

After  conducting  analysis  for  the  first  investigative  question,  4  preferable 
candidate  solutions  were  produced.  The  sensitivity  and  cost-benefit  analysis  used  to 
answer  the  second  and  third  investigate  questions  helped  to  narrow  the  solution  space.  It 
is  found  that  the  alternative  scenario  where  the  optometrist  staffing  level  is  increased  by 
one  and  the  occupational  medicine  doctor  staffing  level  is  decreased  by  one  is  the  best 
choice  among  the  other  alternative  scenarios.  When  implemented,  this  alternative 
scenario  will  improve  the  current  system  in  terms  of  reducing  the  average  total  wait-time 
at  baseline  demand  as  well  as  save  the  system  of  clinics  money  in  all  three  demand 
environments.  However,  it  will  not  be  robust  in  terms  of  the  average  total  wait-time 
when  the  alternative  system  is  faced  with  a  deployment  demand  and  a  medical  incident 
demand. 


Significance  of  Research 

The  711th  Human  Performance  Wing  at  Wright-Patterson  AFB,  OH,  is  sponsoring 
this  research.  The  system  of  clinics  located  at  Wright-Patterson  AFB  has  never  been 
modeled  before  this  research  was  conducted.  This  research  not  only  characterized  the 
behavior  of  the  system  of  clinics,  it  also  provided  insights  about  the  behavior  of  the 
system  of  clinics  when  subjected  to  an  increase  in  patient  demand  both  from  an  increase 
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in  deployments  perspective  as  well  as  from  the  perspective  of  a  mild  medical  incident 


occurring  in  the  local  area.  This  research  can  be  replicated  to  other  healthcare  systems  in 
other  military  installations  to  provide  military  healthcare  decision  makers  insights  into 
the  behavior  of  those  systems  both  in  the  current  patient  demand  and  when  patient 
demand  changes. 


Recommendation  for  Action 

The  recommended  solution  is  to  implement  alternative  scenario  4.  Alternative 
scenario  4  is  implemented  by  adding  an  additional  optometrist  to  the  staff  while 
removing  an  occupational  medicine  doctor  from  the  staff.  This  is  the  recommended 
solution  despite  alternative  scenario  4  not  being  robust  in  deployment  and  medical 
incident  demand  environments.  It  is  found  through  the  cost-benefit  analysis  that 
alternative  scenario  4  is  still  the  best  solution  when  the  system  is  subjected  to  baseline, 
deployment,  and  medical  incident  demands;  statistical  analysis  indicates  that  the 
difference  of  mean  savings  estimate  between  alternative  scenario  4  and  each  of  the  other 
alternative  scenarios  is  statistically  significant  in  terms  of  savings  in  all  three  demand 
environments. 


Recommendations  for  Future  Research 

Future  work  includes  developing  other  alternative  scenarios  that  can  reduce  the 
average  wait-time.  For  example,  reducing  the  average  wait-time  could  be  achieved  by 
increase  the  number  of  examination  rooms  available,  by  alternating  the  process  flow,  by 
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changing  the  current  queuing  strategy  of  a  priority  queue  to  a  first  come  first  served 
queue,  or  by  changing  the  appointment  scheduling  process.  Additional  future  work 
includes  adding  or  removing  more  staff,  performing  the  same  analysis  on  other  military 
installations,  and  doing  an  Air  Force-wide  assessment  to  determine  if  optometry  has  a 
career  field  shortage. 

More  future  work  includes  evaluating  other  alternative  scenarios  beyond  the  five 
studied  and  subjecting  these  to  an  increase  in  patient  demand.  For  example,  reducing  the 
patient  wait-time  could  be  achieved  if  both  the  optometrist  and  flight  medicine  doctor 
increased  staffing  levels  while  reducing  two  or  more  of  the  nurse/technicians  in  an 
increased  patient  and  environment.  Additional  future  work  includes  looking  into 
different  patient  demand  environments.  It  would  be  interesting  to  determine  what  the 
deployment  and  medical  incident  demand  levels  are  at  the  robustness  threshold  as  well  as 
evaluate  other  demand  levels.  The  same  analyses  conduced  in  this  research  should  be 
performed  at  other  military  installations  to  determine  if  their  processes  can  be  improved 
as  well. 


Summary 

This  chapter  evaluates  all  three  investigative  questions  and  insights  are  drawn 
from  the  results  of  the  analyses  to  provide  the  significance  of  the  research.  A  solution  is 
recommended  based  of  the  results  of  the  analyses.  Recommendations  for  future  are 
included  to  investigate  other  scenarios  and  provide  additional  insights  to  the  system  of 
clinics. 
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Appendix  A:  Detailed  Method  in  Establishing  the  Baseline  Model 


Overview 

This  appendix  describes  the  method  to  establish  the  baseline  model  in  full  detail. 
The  first  step  in  developing  a  baseline  model  is  to  formulate  a  conceptual  model  of  the 
system  in  order  to  ensure  that  system  tasks,  resources,  and  work  flows  are  accurately 
captured.  Next,  the  required  data  are  collected  and  fitted  to  probability  distributions.  The 
analysis  of  the  input  data  is  combined  with  the  conceptual  model  of  the  system  into  a  task 
network  that  forms  the  baseline  simulation  model.  This  baseline  simulation  model 
features  the  task  flows,  arrival  rates,  process  probability  distributions,  system  resources 
and  probabilistic  events.  Finally,  the  time  in  system  (TIS)  data  from  the  baseline 
simulation  model  are  validated  against  the  TIS  data  from  the  real  world  system.  This 
method  is  further  described  in  the  subsections  below. 


Step  1:  Conceptual  Model 

The  first  step  in  creating  a  usable  baseline  simulation  model  is  to  understand  the 
system  of  clinics  being  studied.  In  order  to  understand  the  system  of  clinics,  a  conceptual 
model  of  the  daily  operations  is  developed.  To  develop  this  framework,  the  staff 
members  of  the  system  of  clinics  provided  a  general  description  of  daily  operations, 
graphically  depicted  in  Figure  20.  A  typical  daily  operation  starts  when  patients  check  in 
at  the  front  desk  upon  arrival.  Patients  are  given  paperwork  to  fill  out,  if  needed.  Then 
patients  visit  other  stations  where  a  nurse  or  technician  perform  various  tasks  on  them 
(e.g.,  check  vitals/preparation,  laboratory  work,  electrocardiogram  (ECG),  X-ray,  visit 
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other  clinics,  and  additional  visits  to  nurses  or  technicians)  if  they  are  required  prior  to 
visiting  the  doctor.  After  these  preparatory  tasks,  patients  wait  until  the  doctor  is 
available.  When  patients  are  waiting,  the  current  queuing  strategy  of  the  system  of 
clinics  at  WPAFB  is  a  priority  queue:  Patients  with  a  scheduled  appointment  have 
priority  to  see  the  doctor  over  walk-in  patients.  After  visiting  with  the  doctor,  a  follow¬ 
up  appointment  is  scheduled  if  an  additional  visit  is  required.  Patients  then  exit  the 
system  of  clinics. 


Figure  20:  System  of  Clinics  Task  Network 


81 


Step  2:  Data  Collection 

For  each  activity  described  in  the  conceptual  model,  timing  and  decision  data  are 
required  in  order  to  build  the  simulation  model.  These  data  were  collected  by  the  clinic 
medical  staff  by  first  performing  a  trial  data  collection  effort  during  the  month  of  July 
2014  to  become  familiar  with  the  data  collection  process.  The  official  data  collection 
effort  was  conducted  in  August  2014.  Data  were  collected  using  a  clipboard  with  a  clock 
built  into  it  and  a  data  collection  sheet,  see  Figure  21  and  Figure  22.  The  data  collection 
sheet  records  the  following  general  information  about  the  patient’s  visit:  clinic  type 
(audiology,  flight  medicine  clinic,  hearing  conservation  clinic,  occupational  medicine 
clinic,  or  optometry),  patient  type  (military,  civilian,  or  dependent),  status  (scheduled 
appointment  or  walk-in),  date,  appointment  time  (if  applicable),  and  appointment  type. 
The  data  collection  sheet  also  records  the  start  and  end  time  of  each  process  the  patient 
undergoes  which  includes:  patient  check-in,  filling  out  paperwork,  hearing  conservation 
visit,  checking  vitals/preparing  patient,  nurse  or  technician  visit,  laboratory  tasks,  X-ray 
examination,  ECG,  provider  visit,  additional  provider  visits,  and  scheduling  a  follow-up 
appointment.  Annotating  the  start  and  end  times  on  the  sheet  have  negligible  impact  on 
the  performance  of  the  medical  staffs  duties.  A  few  of  the  processes  were  perfonned 
infrequently,  thus  failing  to  provide  an  adequate  number  of  observations  during  the 
August  2014  collection  period.  Thus,  the  data  collection  effort  was  extended  to  include 
the  trial  data  from  July  2014  and  an  additional  collection  from  September  2014  for  these 
infrequent  tasks:  laboratory  tasks,  X-ray  examination,  ECG,  and  visits  to  additional 
nurses  or  technicians.  It  is  reasonable  to  include  some  data  from  the  trial  period  in  July 
2014  for  these  processes  because  the  times  for  these  tasks  were  accurately  collected. 
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DATA  COLLECTION  FOR  PROCESS  IMPROVEMENT 

Patient  ID: 

Clinic: 

Patient  Type:  Military  /  Civilian  /  Dependent  (circle  one) 

Use  back  page  for  additional  comments 

Date: 

Appointment  Time: 

Status:  Walk-In  /  Scheduled  Appointment  (circle  one) 

Appointment  Type: 

Front  Desk  -  Patient  Checking  In 

Time  In  (hour:min:sec): 

Comments: 

Out  (hour:min:sec): 

Staff  Initials: 

Patient  Filling  Out  Paperwork?  Yes  /  No  (circle  one) 

Paperwork  Out  (hour:min:sec): 

Comments: 

Returned  (hour:min:sec): 

Staff  Initials: 

Wait  Time  for  Hearing  Conservation  (If  Applicable) 

Wait  Time  Start  (hour:min:sec): 

Comments: 

Staff  Initials:  i  Wait  Time  End  (hour:min:sec):  Staff  Initials: 

|  Comments: 

i 

i 

i 

i 

i 

i 

Hearing  Conservation?  Yes  /  No  (circle  one) 

Time  In  (hour:min:sec): 

Comments: 

Out  (hour:min:sec): 

Staff  Initials: 

Wait  for  Nurse/Technician  -  Drop  Off/Pick  Up  Medical  Chart 

Dropped  Off  (hour:min:sec): 

Comments: 

Staff  Initials:  i  Picked  Up  (hour:min:sec):  Staff  Initials: 

[Comments: 

i 

i 

i 

i 

i 

i 

Nurse/Technician  -  Checking  Vitals 

(If  multiple  sessions  with  patient,  then  use  reverse  side  for  additional  time  recordings) 

Time  In  (hour:min:sec): 

Comments: 

Out  (hour:min:sec): 

Staff  Initials: 

Lab?  Yes  /  No  (circle  one) 

Lab  Time  In  (hour:min:sec): 

Comments: 

Out  (hour:min:sec): 

Staff  Initials: 

X-Ray?  Yes  /  No  (circle  one) 

X-Ray  Time  In  (hour:min:sec): 

Comments: 

Out  (hour:min:sec): 

Staff  Initials: 

Provider  -  Type  of  Provider  (fill  in  type): 

(If  multiple  sessions  with  patient,  then  use  reverse  side  for  additional  time  recordings) 

Time  In  (hour:min:sec): 

Comments: 

Out  (hour:min:sec): 

Staff  Initials: 

Front  Desk  -  Follow-Up  Appointment?  Yes  /  No  (circle  one) 

Time  In  (hour:min:sec): 

Comments: 

Out  (hour:min:sec): 

Staff  Initials: 

(Use  back  page  for  additional  comments  with  your  initials  next  to  it.) 

Figure  21:  Data  Collection  Sheet  (Front  Page) 
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ADDITIONAL  COMMENTS: 


Nurse/Technician  -  Additional  Sessions  with  Patient 

Session  2- Time  In  (hour:min:sec): 

Out  (hour:min:sec): 

Staff  Initials: 

Session  3- Time  In  (hour:min:sec): 

Out  (hour:min:sec): 

Staff  Initials: 

Session  4- Time  In  (hour:min:sec): 

Out  (hour:min:sec): 

Staff  Initials: 

Comments: 

Additional  Provider  Sessions  with  Patient  -  Type  of  Provider  (fill  in  type): 

Session  2- Time  In  (hour:min:sec): 

Out  (hour:min:sec): 

Staff  Initials: 

Session  3- Time  In  (hour:min:sec): 

Out  (hour:min:sec): 

Staff  Initials: 

Session  4- Time  In  (hour:min:sec): 

Out  (hour:min:sec): 

Staff  Initials: 

Comments: 


Figure  22:  Data  Collection  Sheet  (Back  Page) 
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Step  3:  Input  Analysis 

Upon  completion  of  the  data  collection  effort,  input  data  modeling  was  performed 
on  the  patient  arrivals  and  process  times  in  order  to  form  probability  distributions.  These 
probability  distributions  were  tested  for  independence,  homogeneity,  and  goodness-of-fit. 
All  of  the  final  distributions  in  the  baseline  model  either  successfully  passed  these  tests  or 
were  replaced  by  an  empirical  distribution  directly  representing  the  data.  Table  19 
summarizes  the  frequency  for  each  process  and  possible  patient  path  flows  within  the 
system  of  clinics;  this  information  is  used  to  establish  the  decision  logic  for  the 
simulation  model.  Table  20  summarizes  the  frequency  counts  for  the  clinic  visited, 
patient  type,  and  status  of  the  patient;  these  frequencies  are  used  to  establish  the  decision 
logic  for  the  simulation  model.  Table  21  summarizes  the  likelihood  of  an  optometry 
patient  seeing  the  optometrist  twice  in  a  single  visit;  this  is  unique  from  other  processes 
in  that  the  patient  visits  the  optometrist  again  whereas  the  patient  visits  other  processes 
only  once.  Table  21  is  used  to  establish  the  decision  logic  for  the  simulation  model. 

Table  22  summarizes  the  probability  distribution  for  each  of  the  datasets  being  fitted; 
these  distributions  are  used  in  the  simulation  model  to  determine  the  interarrival  time  for 
each  patient  entering  the  system  as  well  as  process  times  for  process  visited  by  a  patient 
as  they  go  through  the  system. 
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Table  19:  Flight/Occupational  Medicine  Clinic  Process  Frequency  Counts 


Assign  Flight/Occupational  Medicine  Paperwork 


Paperwork  Count  Total  Count  Paperwork 

No  Paperwork  1 

Flight  Medicine 
Occupational  Medicine 

98 

104 

94% 

6% 

95 

179 

53% 

47% 

Need  to  Visit  Hearing  Conservation?  (For  Flight/Occupationa  Medicine  Patients) 


Obs  Count 

Total  Count 

TRUE 

FALSE  ) 

Fligl 

ht  Medicine 

Occupa' 

tional  Medicin 

Need  to  Visit  Nurse  or  Tech  2nd  Time?  (For  Flight/Occupationa  Medicine  Patients) 


1  Obs  Count 

Total  Count 

TRUE 

FALSE  1 

Fligl 

ht  Medicine 

1  i 

104 

1% 

99% 

Occupa' 

tional  Medicin 

1 _ ? _ 

179 

2% 

98% 

Need  to  Visit  Nurse  or  Tech  3rd  Time?  (For  Flight/Occupationa  Medicine  Patients) 


Need  to  See  Optometrist?  (For  Flight/Occupationa  Medicine  Patients) 


1 

1  Obs  Count 

Total  Count 

TRUE 

FALSE  1 

|  See  Optometrist 

1  i 

283 

|  2% 

98%  | 

Need  Follow  Up  Appointment?  (For  Flight/Occupationa  Medicine  Patients) 


Obs  Count 

Total  Count 

TRUE 

FALSE 

1  Schedule  Follow  Up 

17 

283 

6% 

Need  to  Visit  Lab  or  ECG  or  X  Ray? 

Also  Need  ECG  or  X  Ray?  (After  Lab  Assigned) 

1 

Count 

TRUE 

Count 

TRUE 

Lab 

18 

6% 

- ► 

ECG 

2 

11% 

ECG 

2 

1% 

\ 

X  Ray 

1 

6% 

X  Ray 

13 

5% 

\ 

No  Visit  Needed 

15 

No  Visit  Needed 

250 

88% 

\ 

\  Also  Need  X  Ray?  (After  ECG  Assigned) 

* 

Count  TRUE  1 

X  Ray 

1  50% 

No  Visit  Needed 

1  50% 

Table  20:  Patient  Attribute  Frequency  Counts 


Status 

Count 

Percent  1 

Scheduled  Appointment 

474 

91% 

Walk  In 

45 

9% 

Clinic  Visited 

Count 

Percent  1 

Audiology 

68 

13% 

Flight  Medicine 

104 

20% 

Hearing  Conservation 

37 

7% 

Occupational  Medicine 

WE 1 

Optometry 

PatientType  f 

Count 

Percent  1 

Civilian  Employee 

177 

34% 

Dependent 

88 

17% 

Military 

252 

49% 

Table  21:  Optometry  Clinic  Frequency  Count 


See  Optometrist  Twice? 


18 

133 

14% 

86% 

86 


Table  22:  Probability  Distribution  Summary  Table  of  Interarrival/Process  Times  (in 


seconds) 


Create/Process  Node 

Distribution 

Parameters 

K-S  Test 
p- value 

Sample 

Mean 

Sample 
Std.  Dev. 

Arrive  System  of  Clinics 

Weibull 

k  =  0.778 

Lambda  =844 

>0.15 

994 

1550 

Check  In 

Empirical 

N/A 

N/A 

20 

26 

Prepare  to  See  Optometrist  with  Nurse  or  Tech 

Erlang 

ExpMean  =  292 
k  (int)  =  2 

0.0606 

599 

488 

Visit  Optometrist 

Erlang 

ExpMean  =  790 
k  (int)  =  2 

0.0604 

1700 

1100 

Dilation  Effect  Delay 

Weibull 

k  =  0.616 

Lambda  =  1320 

>0.15 

2020 

2680 

Visit  Audiologist 

Weibull 

k  =  1.34 

Lambda  =  1220 

0.131 

1870 

833 

Visit  Hearing  Conservation  Technician 
(Hearing  Conservation  Clinic  Only) 

Exponential 

Mean  =  532 

>0.15 

1080 

495 

Visit  Hearing  Conservation  Technician 
(Non  Hearing  Conservation  Clinics) 

Beta 

Alphal  =  1.79 
Alpha2  =  5.2 

0.136 

861 

328 

Fill  Flight  Medicine  Paperwork 

Weibull 

k  =  1.71 

Lambda  =  311 

>0.15 

289 

155 

Fill  Occupational  Medicine  Paperwork 

Exponential 

Mean  = 454 

>0.15 

463 

472 

Check  Vitals 

Gamma 

Alpha  =  575 
Beta  =  1.34 

0.113 

822 

722 

Perform  Lab 

Erlang 

ExpMean  =  261 
k  (int)  =  2 

0.119 

574 

321 

Perform  ECG 

Exponential 

Mean  =  384 

>0.15 

577 

408 

Perform  X  Ray 

Beta 

Alphal  =  0.926 
Alpha2  =  2.28 

>0.15 

817 

492 

2nd  Session  with  Flight  Medicine  Nurse  or  Tech 

N/A 

Constant  =  377 

N/A 

N/A 

N/A 

2nd  Session  with  Occupational  Medicine  Nurse  or  Tech 

Exponential 

Mean  =  68.8 

>0.15 

548 

90.6 

3rd  Session  with  Flight  Medicine  Nurse  orTech 

N/A 

Constant  =  634 

N/A 

N/A 

N/A 

3rd  Session  with  Occupational  Medicine  Nurse  orTech 

N/A 

Constant  =  818 

N/A 

N/A 

N/A 

See  Flight  Medicine  Physician 

Weibull 

k  =  1.19 

Lambda  =  1170 

>0.15 

1260 

917 

See  Occupational  Medicine  Physician 

Weibull 

k  =  1.49 

Lambda  =801 

>0.15 

984 

480 

Make  Follow  Up  Appointment 

Weibull 

k  =  0.595 

Lambda  =  130 

>0.15 

171 

188 

Test  for  Independence 

The  data  is  tested  for  independence.  This  is  done  to  ensure  that  one  event  does  not 
affect  another.  To  do  this,  the  data  is  tested  for  autocorrelation.  Figure  23  to  Figure  39 
show  the  autocorrelation  plots  for  all  the  input  data.  They  all  indicate  having  no  issues 
with  autocorrelation;  the  values  are  close  to  0.0,  not  greater  than  0.5  and  not  less  than 
-0.5. 
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Autocorrelation  Autocorrelation 


1.0 
0.8 
0.6 
0.4 
0.2 
0.0 
-0.2 
-0.4 
-0.6 
-0.8 
-1 .0 

1  5  10  15  20  25  30  35  40  45  50  55  60  65 


Arrive  System  of  Clinics 


If n— 

.  :  .  Lu  i. 

.  .,,-.1.-,..  ■  ■  ■  i 

1  T  1 

..  a  0  °  1  !  '  |  |  1  1  |  ■  1  1 

II  1 

I'll'  1  ■  "  "  1 

Lag 


Figure  23:  Autocorrelation  Plot  -  Arrive  System  of  Clinics 


Check  In 


Lag 


Figure  24:  Autocorrelation  Plot  -  Check  In 
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Autocorrelation  0  Autocorrelation 


Prepare  to  See  Optometrist  with  Nurse  or  Tech 

1.0 

0.8 

0.6 

0.4 

0.2 - 


-0.2  - - - — — - - 

-0.4 
-0.6 
-0.8 
-1 .0 

- 1 - 1 - 1 - 1 - 1 - 1 - 1 - 1 - 1 - 1 - 1 - 1 - r- 

2  4  6  8  10  12  14  16  18  20  22  24  26 

Lag 


25:  Autocorrelation  Plot  -  Prepare  to  See  Optometrist  with  Nurse  or  Tech 


Visit  Optometrist 


Figure  26:  Autocorrelation  Plot  -  Visit  Optometrist 
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Autocorrelation  Autocorrelation 


Dilation  Effect  Delay 

1.0 

0.8 

0.6 _ 

0.4 

0.2 

0.0  s  1  j 

-0.2 

-0.4 

-0.6  - 

-0.8 
-1 .0 

12  3  4 

Lag 


Figure  27 :  Autocorrelation  Plot  -  Dilation  Effect  Delay 


Visit  Audiologist 


Figure  28:  Autocorrelation  Plot  -  Visit  Audiologist 
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Visit  Hearing  Conservation  Technician  (Hearing  Conservation  Clinic  Only) 


Figure  29:  Autocorrelation  Plot  -  Visit  Hearing  Conservation  Technician  (Hearing 

Conservation  Clinic  Only) 

Visit  Hearing  Conservation  Technician  (Non-Hearing  Conservation  Clinics) 


Figure  30:  Autocorrelation  Plot  -  Visit  Hearing  Conservation  Technician  (Non-Hearing 

Conservation  Clinics) 
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Autocorrelation  Autocorrelation 


Fill  Flight  Medicine  Paperwork 


Figure  3 1 :  Autocorrelation  Plot  -  Fill  Flight  Medicine  Paperwork 


Fill  Occupational  Medicine  Paperwork 


Figure  32:  Autocorrelation  Plot  -  Fill  Occupational  Medicine  Paperwork 
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Autocorrelation  Autocorrelation 


Check  Vitals 


1.0 
0.8 
0.6 
0.4 
0.2 
0.0 
-0.2 
-0.4 
-0.6 
-0.8 
-1 .0 


Figure  33:  Autocorrelation  Plot  -  Check  Vitals 


Perform  Lab 


Lag 


Figure  34:  Autocorrelation  Plot  -  Perform  Lab 
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Autocorrelation  Autocorrelation 


Perform  ECG 


1.0 
0.8 
0.6 
0.4 
0.2 
0.0 
-0.2 
-0.4 
-0.6 
-0.8 
-1 .0 


1 

Lag 


Figure  35:  Autocorrelation  Plot  -  Perform  ECG 


Perform  X  Ray 


Lag 


Figure  36:  Autocorrelation  Plot  -  Perform  X  Ray 
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Autocorrelation  Autocorrelation 


See  Flight  Medicine  Physician 


Lag 


Figure  37:  Autocorrelation  Plot  -  See  Flight  Medicine  Physician 


See  Occupational  Medicine  Physician 


1.0 

0.8 

0.6 

0.4 

0.2 


-0.2  - - 

-0.4 
-0.6 
-0.8 
-1 .0 

1  5  10  15  20  25  30  35  40 


Lag 


Figure  38:  Autocorrelation  Plot  -  See  Occupational  Medicine  Physician 
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Make  Follow  Up  Appointment 


Figure  39:  Autocorrelation  Plot  -  Make  Follow  Up  Appointment 

Test  for  Homogeneity 

Homogeneity  is  a  mathematical  term  used  to  describe  data  that  are  identically 
distributed.  To  test  for  this,  a  visual  inspection  of  the  histogram  is  performed  to  ensure 
that  the  distribution  being  fitted  is  unimodal.  This  indicates  that  the  observations  from 
the  data  collected  come  from  the  same  probability  distribution.  The  histograms  are 
shown  in  each  of  the  input  data  theoretical  distribution  sections. 

Goodness-of-Fit  Test 

A  goodness-of-fit  test  is  used  to  determine  if  the  data  collected  from  the  system  of  clinics 
can  be  represented  by  a  theoretical  distribution  once  the  data  is  tested  for  independence 
and  homogeneity.  This  is  done  by  conducting  a  hypothesis  test.  The  null  hypothesis 
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states  that  the  fitted  theoretical  distribution  is  statistically  similar  to  the  empirical  data. 
The  alternative  hypothesis  states  that  the  fitted  theoretical  distribution  is  not  statistically 
similar  to  the  empirical  data.  The  objective  outcome  of  this  test  is  to  find  a  theoretical 
distribution  that  fails  to  reject  the  null  hypothesis  at  the  confidence  level  of  alpha  being 
0.05  (a  =  0.05).  The  Kolmogorov-Smirnov  test  is  the  type  of  goodness-of-fit  test  that  is 
used  for  this  study.  Since  a  =  0.05,  the  null  hypothesis  is  rejected  if  the  p-value  of  the 
Kolmogorov-Smirnov  test  is  less  than  0.05.  This  concludes  that  the  theoretical 
distribution  being  fitted  is  not  statistically  similar  to  what  is  observed  in  the  data 
collected.  If  the  p-value  of  the  Kolmogorov-Smirnov  test  is  greater  than  or  equal  to  0.05, 
then  the  null  hypothesis  is  not  rejected.  This  concludes  that  the  theoretical  distribution 
being  fitted  is  statistically  similar  to  what  is  observed  in  the  data  collected.  For  our 
purposes,  this  latter  outcome  is  desired. 


Theoretical  Distribution  -  Interarrival  Time 


Figure  40:  Interarrival  Time  -  Histogram 
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Function 

Sq  Error 

Weibull 

0.00181 

Lognormal 

0.00383 

Beta 

0.00767 

Erlang 

0.00926 

Exponential 

0.00926 

Gamma 

0.00951 

Normal 

0.226 

Triangular 

0.412 

Uniform 

0.452 

Figure  41:  Interarrival  Time  -  Fit  All  Summary 


Distribution  Summary 

Distribution:  Weibull 

Expression:  -0.001  +  WEIB(844,  0.778) 

Square  Error:  0.001761 

Chi  Square  Test 

Number  of  intervals 

= 

6 

Degrees  of  freedom 

= 

3 

Test  Statistic 

= 

15.4 

Corresponding  p-value 

< 

0.005 

Kolmogorov- Smirnov  Test 

Test  Statistic 

= 

0.0413 

Corresponding  p-value 

> 

0.15 

Data  Summary 

Number  of  Data  Points 

= 

516 

Min  Data  Value 

= 

0 

Max  Data  Value 

= 

1.99e+004 

Sample  Mean 

= 

994 

Sample  Std  Dev 

= 

1.55e+003 

Histogram  Summary 

Histogram  Range 

= 

-0.001  to  1. 99e+004 

Humber  of  Intervals 

= 

22 

Figure  42:  Interarrival  Time  -  Weibull  Distribution  Summary 
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Theoretical  Distribution  -  Check  In  Time 


Figure  43:  Check  In  Time  -  Histogram 


Table  23:  Check  In  Time  -  Empirical  Distribution  Summary 


Full  ARENA  Expression 

CONT  (0,  0,  0.0119521912350598,  1,  0.0318725099601594,  3,  0.099601593625498,  4,  0.187250996015936,  5, 
0.250996015936255,  6,  0.332669322709163,  7,  0.394422310756972,  8,  0.448207171314741,  9, 
0.48804780876494,  10,  0.543824701195219,  11,  0.579681274900398,  12,  0.603585657370518,  13, 
0.635458167330677,  14,  0.649402390438247,  15,  0.663346613545817,  16,  0.689243027888446,  17, 
0.703187250996016,  18,  0.719123505976096,  19,  0.743027888446215,  20,  0.754980079681275,  21, 
0.764940239043825,  22,  0.772908366533865,  23,  0.778884462151394,  24,  0.788844621513944,  25, 
0.798804780876494,  26,  0.808764940239044,  27,  0.812749003984064,  28,  0.816733067729084,  29, 
0.830677290836653,  30,  0.834661354581673,  31,  0.844621513944223,  32,  0.846613545816733,  34, 
0.852589641434263,  36,  0.860557768924303,  37,  0.866533864541833,  38,  0.876494023904382,  39, 
0.882470119521912,  40,  0.888446215139442,  41,  0.896414342629482,  42,  0.904382470119522,  43, 
0.906374501992032,  44,  0.910358565737052,  49,  0.912350597609562,  50,  0.914342629482072,  51, 
0.918326693227092,  53,  0.924302788844621,  54,  0.926294820717131,  55,  0.928286852589641,  56, 
0.930278884462151,  58,  0.932270916334661,  59,  0.940239043824701,  60,  0.942231075697211,  61, 
0.944223107569721,  62,  0.946215139442231,  64,  0.948207171314741,  65,  0.950199203187251,  67, 
0.952191235059761,  68,  0.954183266932271,  72,  0.958167330677291,  76,  0.962151394422311,  81, 
0.964143426294821,  90,  0.966135458167331,  93,  0.968127490039841,  94,  0.970119521912351,  95, 
0.972111553784861,  100,  0.97609561752988,  102,  0.97808764940239,  107,  0.9800796812749,  113, 
0.98207171314741,  114,  0.98605577689243,  118,  0.98804780876494,  120,  0.99003984063745,  134, 
0.99203187250996,  144,  0.99402390438247,  154,  0.99601593625498,  167,  0.99800796812749,  170,  1,  180) 


mean  =  20 
std.  dev.  =  26 
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Theoretical  Distribution  -  Prepare  to  See  Optometrist  with  Nurse  or  Tech  Time 


Figure  44:  Prepare  to  See  Optometrist  with  Nurse  or  Tech  Time  -  Histogram 


Function 

Sq  Error 

Erlang 

0.0175 

Weibull 

0.0229 

Gamma 

0.0232 

Beta 

0.0428 

Normal 

0.0477 

Exponential 

0.064 

Lognormal 

0.0913 

Triangular 

0.3 

Uniform 

0.324 

Figure  45:  Prepare  to  See  Optometrist  with  Nurse  or  Tech  Time  -  Fit  All  Summary 
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Distribution  Summary 

Distribution:  Erlang 

Expression:  15  +  ERLA(292,  2) 

Square  Error:  0.017503 

Shi  Square  Test 

Number  of  intervals 

= 

3 

Degrees  of  freedom 

= 

0 

Test  Statistic 

= 

9.93 

Corresponding  p-value 

< 

0.005 

<o lmogo r o v- Smi r no v  Test 

Test  Statistic 

= 

0.128 

Corresponding  p-value 

= 

0.0606 

Data  Summary 

dumber  of  Data  Points 

= 

105 

Hin  Data  Value 

= 

15 

Hax  Data  Value 

= 

4 . 75e+003 

Sample  Mean 

= 

599 

Sample  Std  Dev 

= 

488 

Histogram  Summary 

Histogram  Range 

= 

15  to  4 . 75e+003 

dumber  of  Intervals 

= 

10 

Figure  46:  Prepare  to  See  Optometrist  with  Nurse  or  Tech  Time  -  Erlang  Distribution 

Summary 


Theoretical  Distribution  -  Visit  Optometrist  Time 


Figure  47:  Visit  Optometrist  Time  -  Histogram 
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Function 

Sq  Error 

Erlang 

0.0137 

Weibull 

0.017 

Gamma 

0.0191 

Normal 

0.0218 

Beta 

0.0228 

Exponential 

0.0795 

Triangular 

0.0874 

Lognormal 

0.0948 

Uniform 

0.155 

Figure  48:  Visit  Optometrist  Time  -  Fit  All  Summary 


Distribution  Summary 

Distribution:  Erlang 

Expression:  120  +  ERLA(790,  2) 

Square  Error:  0.013685 

Chi  Square  Test 

Number  of  intervals 

= 

6 

Zegrees  of  freedom 

= 

3 

Test  Statistic 

= 

13.1 

Corresponding  p-value 

< 

0.005 

Kolmogorov- Smirnov  Test 

Test  Statistic 

= 

0.107 

Corresponding  p-value 

= 

0.0604 

Data  Summary 

Number  of  Data  Points 

= 

151 

Min  Data  Value 

= 

120 

Max  Data  Value 

= 

8.26e+003 

Sample  Mean 

= 

1. 7e+003 

Sample  Std  Dev 

= 

l.le+003 

Histogram  Summary 

Histogram  Range 

= 

120  to  8.26e+003 

Number  of  Intervals 

= 

12 

Figure  49:  Visit  Optometrist  Time  -  Erlang  Distribution  Summary 
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Theoretical  Distribution  -  Dilation  Effect  Delay  Time 


Figure  50:  Dilation  Effect  Delay  Time  -  Histogram 


Function 

Sq  Error 

Heibull 

0.025 

Exponential 

0.0489 

Erlang 

0.0489 

Gamma 

0.0845 

Beta 

0.107 

Lognormal 

0.217 

Normal 

0.364 

Triangular 

0.448 

Uniform 

0.545 

Figure  51:  Dilation  Effect  Delay  Time  -  Fit  All  Summary 
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Distribution  Summary 
Distribution:  Keibull 

Expression:  300  +  WEIB(1.32e+003,  0.616) 

Square  Error:  0.025049 


Kolmogorov-Smirnov  Test 

Test  Statistic  =0.26 

Corresponding  p-value  >  0.15 

Data  Summary 

Number  of  Data  Points 
Min  Data  Value 
Max  Data  Value 
Sample  Mean 
Sample  Std  Dev 

Histogram  Summary 

Histogram  Range  =  300  to  1.09e+004 

Humber  of  Intervals  =  5 


=  14 
=  300 

=  1.09e+004 
=  2.02e+003 
=  2.68e+003 


Figure  52:  Dilation  Effect  Delay  Time  -  Weibull  Distribution  Summary 


Theoretical  Distribution  -  Visit  Audiologist  Time 


Figure  53:  Visit  Audiologist  Time  -  Histogram 
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Function 

Sq  Error 

Normal 

0.037 

Weibull 

0.0447 

Beta 

0.0549 

Gamma 

0.0583 

Exponential 

0.104 

Erlang 

0.104 

Triangular 

0.105 

Lognormal 

0.176 

Uniform 

0.181 

Figure  54:  Visit  Audiologist  Time  -  Fit  All  Summary 


Distribution  Summary 

Distribution:  Weibull 

expression:  736  +  WEIB (1. 22e+003,  1.34) 

Square  Error:  0.044677 

Shi  Square  Test 

Number  of  intervals 

= 

4 

Degrees  of  freedom 

= 

1 

Test  Statistic 

= 

16.2 

Corresponding  p-value 

< 

0.005 

Kolmogorov- Smirnov  Test 

Test  Statistic 

= 

0.143 

Corresponding  p-value 

= 

0.131 

Data  Summary 

dumber  of  Data  Points 

= 

65 

din  Data  Value 

= 

736 

Hax  Data  Value 

= 

5.61e+003 

Sample  Mean 

= 

1. 87e+003 

Sample  Std  Dev 

= 

833 

Histogram  Summary 

Histogram  Range 

= 

736  to  5. 61e+003 

Number  of  Intervals 

= 

8 

Figure  55:  Visit  Audiologist  Time  -  Weibull  Distribution  Summary 
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Theoretical  Distribution  -  Visit  Hearing  Conservation  Technician  (Hearing 
Conservation  Clinic  Only)  Time 


Figure  56:  Visit  Hearing  Conservation  Technician  (Hearing  Conservation  Clinic  Only) 

Time  -  Histogram 


Function 

Sq  Error 

Erlang 

0.00943 

Exponential 

0.00943 

Weibull 

0.0113 

Gamma 

0.0127 

Beta 

0.0207 

Lognormal 

0.0469 

Normal 

0.0682 

Triangular 

0.0745 

Uniform 

0.163 

Figure  57:  Visit  Hearing  Conservation  Technician  (Hearing  Conservation  Clinic  Only) 

Time  -  Fit  All  Summary 
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Distribution  Summary 

Distribution:  Exponential  | 

Expression:  547  +  EXPO (532) 

Square  Error:  0.009430 

Chi  Square  Test 

Number  of  intervals 

= 

3 

Degrees  of  freedom 

= 

1 

Test  Statistic 

= 

0.58 

Corresponding  p-value 

= 

0.464 

Kolmogorov- Smirnov  Test 

Test  Statistic 

= 

0.0851 

Corresponding  p-value 

> 

0.15 

Data  Summary 

Number  of  Data  Points 

= 

37 

Min  Data  Value 

= 

547 

Max  Data  Value 

= 

2. 73e+003 

Sample  Mean 

= 

1. 08e+003 

Sample  Std  Dev 

= 

495 

Histogram  Summary 

Histogram  Range 

= 

547  to  2 . 73e+003 

Number  of  Intervals 

= 

6 

Figure  58:  Visit  Hearing  Conservation  Technician  (Hearing  Conservation  Clinic  Only) 
Time  -  Exponential  Distribution  Summary 
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Theoretical  Distribution  -  Visit  Hearing  Conservation  Technician  (Non  Hearing 


Conservation  Clinics)  Time 


Figure  59:  Visit  Hearing  Conservation  Technician  (Non  Hearing  Conservation  Clinics) 

Time  -  Histogram 


Function 

Sq  Error 

Normal 

0.0572 

Beta 

0.0613 

Weibull 

0.0699 

Erlang 

0.0713 

Gamma 

0.0974 

Triangular 

0.128 

Exponential 

0.181 

Uniform 

0.233 

Lognormal 

0.258 

Figure  60:  Visit  Hearing  Conservation  Technician  (Non  Hearing  Conservation  Clinics) 

Time  -  Fit  All  Summary 
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Distribution  Summary 

Distribution:  Beta 

Expression:  317  +  2 

12e+003  *  BETA(1.79,  5.2) 

Square  Error:  0.061334 

Chi  Square  Test 

Number  of  intervals 

= 

4 

Degrees  of  freedom 

= 

1 

Test  Statistic 

= 

13 

Corresponding  p-value 

< 

0.005 

Kolmogorov- Smirnov  Test 

Test  Statistic 

= 

0.146 

Corresponding  p-value 

= 

0.136 

Data  Summary 

Number  of  Data  Points 

= 

61 

Min  Data  Value 

= 

317 

Max  Data  Value 

= 

2. 44e+003 

Sample  Mean 

= 

861 

Sample  Std  Dev 

= 

328 

Histogram  Summary 

Histogram  Range 

= 

317  to  2. 44e+003 

Number  of  Intervals 

= 

7 

Figure  61:  Visit  Hearing  Conservation  Technician  (Non  Hearing  Conservation  Clinics) 

Time  -  Beta  Distribution  Summary 


Theoretical  Distribution  -  Fill  Flight  Medicine  Paperwork  Time 


Figure  62:  Fill  Flight  Medicine  Paperwork  Time  -  Histogram 
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Function 

Sq  Error 

Weibull 

0.0103 

Beta 

0.0149 

Triangular 

0.0149 

Erlang 

0.0159 

Gamma 

0.0167 

Normal 

0.0176 

Uniform 

0.0644 

Exponential 

0.0717 

Lognormal 

0.0872 

Figure  63:  Fill  Flight  Medicine  Paperwork  Time  -  Fit  All  Summary 


Distribution  Summary 

Distribution:  Keibull 

Expression:  7  +  WEIB(311,  1.71) 

Square  Error:  0.010305 

Shi  Square  Test 

Number  of  intervals 

= 

5 

Degrees  of  freedom 

= 

2 

Test  Statistic 

= 

5.02 

Corresponding  p-value 

= 

0.0852 

■Colmogorov-Smirnov  Test 

Test  Statistic 

= 

0.0866 

Corresponding  p-value 

> 

0.15 

Data  Summary 

dumber  of  Data  Points 

= 

91 

4in  Data  Value 

= 

7 

iax  Data  Value 

= 

756 

Sample  Mean 

= 

289 

Sample  Std  Dev 

= 

155 

Histogram  Summary 

Sistogram  Range 

= 

7  to  756 

dumber  of  Intervals 

= 

9 

Figure  64:  Fill  Flight  Medicine  Paperwork  Time  -  Weibull  Distribution  Summary 
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Theoretical  Distribution  -  Fill  Occupational  Medicine  Paperwork  Time 


Figure  65:  Fill  Occupational  Medicine  Paperwork  Time  -  Histogram 


Function 

Sq  Error 

Erlang 

0.0138 

Exponential 

0.0138 

Weibull 

0.014 

Gamma 

0.0148 

Lognormal 

0.0265 

Beta 

0.0275 

Komal 

0.12 

Triangular 

0.138 

Uniform 

0.211 

Figure  66:  Fill  Occupational  Medicine  Paperwork  Time  -  Fit  All  Summary 
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Distribution  Summary 

Distribution:  Exponential 

Expression:  9  +  EXPO (454) 

Square  Error:  0.013758 

Chi  Square  Test 

Number  of  intervals 
Degrees  of  freedom 
Test  Statistic 
Corresponding  p-value 

Kolmogorov- Smirnov  Test 

Test  Statistic  =  0.116 

Corresponding  p-value  >  0.15 

Data  Summary 

Number  of  Data  Points 
Min  Data  Value 
Max  Data  Value 
Sample  Mean 
Sample  Std  Dev 

Histogram  Summary 

Histogram  Range  =  9  to  2.41e+003 

Number  of  Intervals  =  9 


=  89 
=  9 

=  2. 41e+003 
=  463 
=  472 


=  4 
=  2 

=  5.54 
=  0.0662 


Figure  67:  Fill  Occupational  Medicine  Paperwork  Time  -  Exponential  Distribution 

Summary 


Theoretical  Distribution  -  Check  Vitals  Time 


Figure  68:  Check  Vitals  Time  -  Histogram 
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Function 

Sq  Error 

Gamma 

0.00745 

Weibull 

0.00975 

Erlang 

0.0209 

Exponential 

0.0209 

Lognormal 

0.0243 

Beta 

0.0291 

Normal 

0.0443 

Triangular 

0.0735 

Uniform 

0.113 

Figure  69:  Check  Vitals  Time  -  Fit  All  Summary 


Distribution  Summary 

Distribution:  Gamma 

Expression:  51  +  GAMM(575#  1.34) 

Square  Error:  0.007450 

Chi  Square  Test 

Number  of  intervals 

= 

8 

Degrees  of  freedom 

= 

5 

Test  Statistic 

= 

17.4 

Corresponding  p-value 

< 

0.005 

Kolmogorov- Smirnov  Test 

Test  Statistic 

= 

0.0737 

Corresponding  p-value 

= 

0.113 

Data  Summary 

dumber  of  Data  Points 

= 

262 

Min  Data  Value 

= 

51 

Max  Data  Value 

= 

4 . 17e+003 

Sample  Mean 

= 

822 

Sample  Std  Dev 

= 

722 

Histogram  Summary 

Histogram  Range 

= 

51  to  4 . 17e+003 

Humber  of  Intervals 

= 

16 

Figure  70:  Check  Vitals  Time  -  Distribution  Summary 


113 


Theoretical  Distribution  -  Perform  Lab  Time 


Figure  71:  Perform  Lab  Time  -  Histogram 


Function 

Sq  Error 

Normal 

0.184 

Heibull 

0.353 

Beta 

0.402 

Triangular 

0.431 

Uniform 

0.521 

Gamma 

0.523 

Erlang 

0.523 

Exponential 

0.595 

Lognormal 

0.619 

Figure  72:  Perform  Lab  Time  -  Fit  All  Summary 


114 


Distribution  Summary 


Distribution: 
Expression: 
Square  Error: 


Erlang 

52  +  ERLA(261,  2) 
0.021242 


Dhi  Square  Test 


Number  of  intervals 

5 

Degrees  of  freedom 

= 

2 

Test  Statistic 

= 

9.1 

Corresponding  p-value 

= 

0.0109 

Kolmogorov- Smirnov  Test 

Test  Statistic 

= 

0.129 

Corresponding  p-value 

= 

0.119 

Data  Summary 

dumber  of  Data  Points 

= 

83 

Hin  Data  Value 

= 

52 

Kax  Data  Value 

= 

1. 65e+003 

Sample  Mean 

= 

574 

Sample  Std  Dev 

= 

321 

Histogram  Summary 

Histogram  Range  =  52  to  1.65e+003 

dumber  of  Intervals  =  8 


Figure  73:  Perform  Lab  Time  -  Erlang  Distribution  Summary 


Theoretical  Distribution  -  Perform  ECG  Time 


Figure  74:  Perform  ECG  Time  -  Histogram 
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Function 

Sq  Error 

Erlang 

0.0225 

Exponential 

0.0225 

Weibull 

0.0358 

’aairma 

0.038 

Eeta 

0.0768 

Lognormal 

0.0989 

Normal 

0.131 

Triangular 

0.165 

Uniform 

0.276 

Figure  75:  Perform  ECG  Time  -  Fit  All  Summary 


Distribution  Summary 


Distribution: 
Expression: 
Square  Error: 


Exponential 
193  +  EXPO (384) 
0.022540 


Kolmogorov-Smirnov  Test 

Test  Statistic 

= 

0.229 

Corresponding  p-value 

> 

0.15 

Data  Summary 

Number  of  Data  Point3 

= 

15 

Min  Data  Value 

= 

193 

Max  Data  Value 

= 

1. 9e+003 

Sample  Mean 

= 

577 

Sample  Std  Dev 

= 

4oe 

Histogram  Summary 

Histogram  Range 

= 

193  to  1.9e+003 

Number  of  Intervals 

= 

5 

Figure  76:  Perform  ECG  Time  -  Exponential  Distribution  Summary 
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Theoretical  Distribution  -  Perform  X  Ray  Time 


Figure  77:  Perform  X  Ray  Time  -  Histogram 


Function 

Sq  Error 

Beta 

0.0172 

Weibull 

0.0176 

Exponential 

0.0224 

Erlang 

0.0224 

Gamma 

0.0262 

Triangular 

0.0464 

Normal 

0.0477 

Lognormal 

0.0759 

Uniform 

0.137 

Figure  78:  Perform  X  Ray  Time  -  Fit  All  Summary 
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Distribution  Summary 

Distribution:  Beta 

Expression:  175  +  2 

23e+003  *  BETA(0. 926,  2.28) 

Square  Error:  0.017191 

Chi  Square  Test 

Number  of  intervals 

= 

3 

Degrees  of  freedom 

= 

0 

Test  Statistic 

= 

2.75 

Corresponding  p-value 

< 

0.005 

Kolmogorov-Smirnov  Test 

Test  Statistic 

= 

0.161 

Corresponding  p-value 

> 

0.15 

Data  Summary 

Number  of  Data  Points 

- 

28 

Min  Data  Value 

= 

175 

Max  Data  Value 

= 

2. 4e+003 

Sample  Mean 

= 

817 

Sample  Std  Dev 

= 

492 

Histogram  Summary 

Histogram  Range 

= 

175  to  2. 4e+003 

Number  of  Intervals 

= 

5 

Figure  79:  Perform  X  Ray  Time  -  Beta  Distribution  Summary 


Theoretical  Distribution  -  2nd  Session  with  Occupational  Medicine  Nurse  or  Tech 
Time 


Figure  80:  2nd  Session  with  Occupational  Medicine  Nurse  or  Tech  Time  -  Histogram 
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Function 

Sq  Error 

Erlang 

0.0709 

Exponential 

0.0709 

Gamma 

0.0855 

Weibull 

0.108 

Triangular 

0.125 

Lognormal 

0.135 

Beta 

0.153 

Uniform 

0.175 

Normal 

0.196 

Figure  81:  2nd  Session  with  Occupational  Medicine  Nurse  or  Tech  Time  -  Fit  All 

Summary 


Distribution  Summary 

Distribution:  Exponential 

Expression:  479  +  EXPO (68. 8) 

Square  Error:  0.070854 


Kolmogorov- Smirnov  Test 

Test  Statistic  =  0.301 

Corresponding  p-value  >0.15 

Data  Summary 

Number  of  Data  Points 
Min  Data  Value 
Max  Data  Value 
Sample  Mean 
Sample  Std  Dev 

Histogram  Summary 

Histogram  Range  =  479  to  681 

Number  of  Intervals _ =  5 _ 


=  4 
=  479 
=  681 
=  548 
=  90.6 


Figure  82:  2nd  Session  with  Occupational  Medicine  Nurse  or  Tech  Time  -  Exponential 

Distribution  Summary 
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Theoretical  Distribution  -  See  Flight  Medicine  Physician  Time 


Figure  83:  See  Flight  Medicine  Physician  Time  -  Histogram 


Function 

Sq  Error 

Weibull 

0.0125 

Ganna 

0.0165 

Beta 

0.0193 

Erlang 

0.029 

Exponential 

0.029 

Normal 

0.044 

Lognormal 

0.0737 

Triangular 

0.121 

Uniform 

0.161 

Figure  84:  See  Flight  Medicine  Physician  Time  -  Fit  All  Summary 
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Distribution  Summary 

Distribution:  Weibull 

Expression:  146  +  WEIB (1. 17e+003,  1.19) 

Square  Error:  0.012487 

Chi  Square  Test 

Number  of  intervals 

= 

4 

Degrees  of  freedom 

= 

1 

Test  Statistic 

= 

3.89 

Corresponding  p-value 

= 

0.049 

Kolmogorov-Smirnov  Test 

Test  Statistic 

= 

0.105 

Corresponding  p-value 

> 

0.15 

Data  Summary 

Number  of  Data  Points 

= 

88 

Min  Data  Value 

= 

146 

Max  Data  Value 

= 

5.71e+003 

Sample  Mean 

= 

1. 26e+003 

Sample  Std  Dev 

= 

917 

Histogram  Summary 

Histogram  Range 

= 

146  to  5.71e+003 

Number  of  Intervals 

= 

9 

Figure  85:  See  Flight  Medicine  Physician  Time  -  Distribution  Summary 


Theoretical  Distribution  -  See  Occupational  Medicine  Physician  Time 


Figure  86:  See  Occupational  Medicine  Physician  Time  -  Histogram 
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Function 

Sq  Error 

Weibull 

0.0056 

Beta 

0.00613 

Erlang 

0.00627 

Normal 

0.00912 

Gamma 

0.00961 

Exponential 

0.0446 

Lognormal 

0.0504 

Triangular 

0.0743 

Uniform 

0.0942 

Figure  87:  See  Occupational  Medicine  Physician  Time  -  Fit  All  Summary 


Distribution  Summary 

Distribution:  Weibull 

Expression:  253  +  WEIB(801,  1.49) 

Square  Error:  0.005601 

Chi  Square  Test 

Number  of  intervals 

= 

7 

Degrees  of  freedom 

= 

4 

Test  Statistic 

= 

8.73 

Corresponding  p-value 

= 

0.0722 

Kolmogorov- Smirnov  Test 

Test  Statistic 

= 

0.0554 

Corresponding  p-value 

> 

0.15 

Data  Summary 

Number  of  Data  Points 

— 

174 

Min  Data  Value 

= 

253 

Max  Data  Value 

= 

3. 42e+003 

Sample  Mean 

= 

984 

Sample  Std  Dev 

= 

480 

Histogram  Summary 

Histogram  Range 

= 

253  to  3 . 42e+003 

Number  of  Intervals 

= 

13 

Figure  88:  See  Occupational  Medicine  Physician  Time  -  Weibull  Distribution  Summary 
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Theoretical  Distribution  -  Make  Follow  Up  Appointment  Time 


Figure  89:  Make  Follow  Up  Appointment  Time  -  Histogram 


Function 

Sq  Error 

Weibull 

0.00861 

Gamma 

0.0137 

Erlang 

0.0184 

Exponential 

0.0184 

Lognormal 

0.0373 

Beta 

0.109 

Triangular 

0.135 

Normal 

0.16 

Uniform 

0.198 

Figure  90:  Make  Follow  Up  Appointment  Time  -  Fit  All  Summary 
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Distribution  Summary 

Distribution:  Weibull 

Expression:  -0.001  +  WEIB(130,  0.S95) 

Square  Error:  0.008613 


Kolmogorov-Smirnov  Test 

Test  Statistic  =  0.265 

Corresponding  p- value  >  0.15 

Data  Summary 

Number  of  Data  Points 
Min  Data  Value 
Max  Data  Value 
Sample  Mean 
Sample  Std  Dev 

Histogram  Summary 

Histogram  Range  =  -0.001  to  628 

Number  of  Intervals  =  5 


=  17 
=  0 
=  628 
=  171 
=  188 


Figure  91:  Make  Follow  Up  Appointment  Time  -  Distribution  Summary 


Step  4:  Arena  Model 

The  input  data  described  above  are  combined  with  the  process  flows  to  form  a 
task  network.  Figure  20  in  Step  1  provides  a  task  network  which  is  a  visual 
representation  of  the  conceptual  model.  The  conceptual  model  is  translated  into  a  task 
network  by  representing  decision  logic  as  decision  nodes,  processes  as  task  nodes,  and 
the  order  of  these  tasks  and  decisions  as  directional  arcs.  When  a  patient  goes  through 
the  system,  the  individual  will  process  through  the  various  nodes  established  in  the  task 
network  and  will  follow  the  decision  logic  throughout  the  model.  For  example,  all 
patients  go  through  the  check-in  node,  followed  by  decision  logic  to  determine  (1)  if  the 
patient  needs  to  fill  out  paperwork  and  (2)  which  clinic  the  patient  will  visit: 
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Audiology/Hearing  Conservation  Clinics,  Optometry  Clinic,  or  Flight/Occupational 
Medicine  Clinics. 

If  the  assigned  clinic  is  Audiology  or  Hearing  Conservation,  the  task  flow  is 
simple  in  that  the  patient  will  go  to  the  respective  node  of  Visit  Doctor/Specialist  to 
process  through  a  visit  with  the  audiologist  or  hearing  conservation  technician.  Once 
completed,  the  patient  exits  the  system. 

If  the  assigned  clinic  is  Optometry,  then  the  optometry  nurse  or  technician 
prepares  the  patient  to  see  the  optometrist.  The  patient  then  visits  the  optometrist. 
Decision  logic  is  used  to  determine  if  the  patient  needs  to  see  the  optometrist  again.  If 
the  patient  needs  to  see  the  optometrist  again,  it  is  because  their  eyes  need  to  be  dilated 
for  examination.  The  patient  waits  for  the  dilation  drug  to  take  effect  before  visiting  the 
optometrist  a  second  time;  the  patient  waiting  for  the  dilation  drug  to  take  effect  is 
counted  as  value-added  time  and  not  attributed  to  wait-time  because  this  is  a  necessary 
process.  Once  completed,  the  patient  exits  the  system. 

If  the  assigned  clinic  is  Flight  Medicine  or  Occupational  Medicine,  the  nurse  or 
technician  of  their  respective  clinic  checks  the  vitals  of  the  patient.  A  series  of  decision 
nodes  is  created  to  determine  if  the  patient  needs  to  perform  various  tasks.  If  the  patient 
needs  to  perform  a  laboratory  task,  ECG,  X-ray,  see  a  nurse  or  technician  numerous 
times,  visit  hearing  conservation  technician,  and/or  visit  the  optometrist  once,  then  the 
patient  will  go  to  the  needed  process  nodes  in  any  order  based  upon  availability:  if  the 
process  node  is  using  up  all  the  resources  to  perform  the  task,  then  the  patient  proceeds  to 
the  next  task  and  returns  to  the  previous  node  when  it  becomes  available.  When 
completed,  the  patient  will  visit  the  doctor  of  the  respective  clinic.  A  decision  node 
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determines  if  the  patient  makes  a  follow  up  appointment.  Once  completed,  the  patient 


exits  the  system. 

In  addition  to  capturing  the  process  flows,  decision  logic,  and  timing  data,  the  clinic 
staff  also  annotated  the  type  and  quantity  of  resources  used.  Table  24  summarizes  the 
resources  used  in  the  system  of  clinics.  There  are  unique  characteristics  associated  with  a 
few  of  the  resources.  These  characteristics  are  listed  here: 

•  There  is  a  front  desk  station  at  the  entrance  of  the  building  that  can  only  be 
manned  by  one  administration  technician. 

•  The  Hearing  Conservation  technician  is  being  treated  as  a  provider  for  this  clinic. 

•  The  Flight  Medicine  and  Occupational  Medicine  clinics  share  6  examination 
rooms. 

•  The  laboratory  and  ECG  rooms  are  operated  by  the  nurse  or  technicians  of  either 
the  Flight  Medicine  or  Occupational  Medicine  clinic,  depending  on  which  clinic 
the  patient  belongs  to. 

•  The  ECG  room  is  co-located  with  one  of  the  optometry  examination  rooms. 

•  The  X-Ray  room  is  manned  by  an  X-Ray  technician. 

Once  the  resources  were  incorporated  into  the  model,  it  was  then  validated. 


Table  24:  System  of  Clinic  Resources 


Rooms 

Providers 

Nurse/Tech 

Front  Desk  Station 

1 

Not  Applicable 

1 

Audiology 

3 

2 

0 

Flight  Medicine 

shared  6  with  occupational  medicine 

4 

shared  8  with  Lab  and  ECG 

Hearing  Conservation 

1 

1 

Not  Applicable 

Occupational  Medicine 

shared  6  with  flight  medicine 

4 

shared  4  with  Lab  and  ECG 

Optometry 

1  dedicated  to  optometry  with  1  shared  with  ECG 

1 

1 

Lab 

2 

Not  Applicable 

performed  by  respective  clinic's  nurse/tech 

ECG 

1  shared  with  optometry 

Not  Applicable 

performed  by  respective  clinic's  nurse/tech 

X  Ray 

1 

Not  Applicable 

1 
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Step  5:  Validation 

Validation  is  an  important  step  in  creating  a  baseline  simulation  model.  It 
provides  statistical  evidence  that  the  model  adequately  reflects  the  real  world  system.  For 
satisfactory  validation,  a  confidence  interval  of  10%  within  the  mean  is  desired.  For  this 
system,  the  average  time  in  system  is  54.13  minutes,  thus  a  half-width  of  5.4  min  or  less 
is  required.  A  99%  confidence  interval  for  this  system  produced  a  half-width  of  3.16 
minutes,  thus  a  99%  confidence  interval  level  was  deemed  sufficient  for  use  in  validation. 
The  tradeoff  in  using  a  99%  confidence  interval  level  is  that  it  provides  a  higher  level  of 
confidence  at  the  risk  of  an  unacceptably  large  half-width.  Because  the  99%  confidence 
interval  level  has  a  half-width  that  is  considerably  less  than  the  desired  +10%  of  the 
mean,  the  99%  confidence  interval’s  half-width  is  deemed  to  be  acceptably  narrow. 

Upon  establishing  a  confidence  interval  level,  real  world  data  is  compared  to 
simulation  data  using  a  99%  confidence  interval.  In  order  to  determine  the  number  of 
replications  needed  to  run  the  model,  an  approximation  equation  is  used  [11]: 

n  =  no  (2) 

Where  n  is  the  number  of  replications  needed,  n0  is  the  number  of  replications  in  the 
initial  production  run,  h0  is  the  half-width  of  the  initial  production  run,  and  h  is  the 
desired  half-width.  An  initial  run  on  the  model  is  conducted  with  n0=10  as  an  arbitrary 
initial  number  of  replications.  It  produces  an  initial  half-width  of  h0  =  6.03  min.  Based 
on  the  desired  half- width  of  h  —  3.16  min  (taken  from  the  real  world  half-width  of  3.16 
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minutes),  an  estimate  on  the  number  of  replications  needed,  n,  is  evaluated;  first  iteration: 
n=10(6.032/3.162)  =  36.49.  This  process  is  repeated  three  more  times  to  determine  a 
reasonable  number  of  replications;  second  iteration:  37(3.3 12/3.162)  =  40.68;  third 
iteration:  41(3.21 2/3 . 1 62)  =  42.39;  fourth  iteration:  43(3.092/3.162)  =  41.20.  It  is 
determined  that  at  least  41.20,  rounded  up  (to  be  conservative)  to  42  replications,  is 
required  to  achieve  the  desired  half-width.  The  confidence  intervals  of  the  real  world 
data  and  simulation  data  reveal  that  there  is  no  statistical  difference  between  the  model 
and  the  real  world  system,  see  Figure  92.  This  is  demonstrated  by  the  overlap  of  both 
confidence  intervals,  thus  validating  the  baseline  model.  Note  that  the  average  time  in 
system  of  the  simulation  is  49.29  minutes.  This  indicates  that  the  simulation  is,  on 
average,  4.84  minutes  faster  than  reality.  To  account  for  this  difference,  it  is 
hypothesized  that  the  exclusion  of  transit  time  in  the  model  is  what  is  causing  a  slightly 
faster  time  in  system.  This  is  of  negligible  concern  for  the  purpose  of  this  study  as  the 
patient  wait-time  is  the  focus  of  this  investigation,  not  the  total  time  in  system,  and  transit 
time  is  not  expected  to  impact  wait-time. 
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overlap 


50.97  57.28 

■ - a - 

46.12  52.45 


♦  Reality  99%C.I. 

■  Simulation  99%  C.l. 


45  50  55  60 

Time  in  System  (Minutes) 


Figure  92:  Real-World  versus  Simulation  TIS  with  a  99%  Confidence  Interval 
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Model  Limitations  and  Assumptions 


Table  25:  Model  Limitations  and  Assumptions 


Model 

Limitations/Assumptions 

Reasoning 

Miscellaneous  activities 

are  excluded 

During  the  data  collecting  activity,  mail  delivery  personnel  is  observed 
to  interact  with  the  administration  technician  but  are  not  included  in 

the  model  for  two  reasons: 

1. )  They  are  not  considered  a  patient  in  the  system. 

2. )  It  only  took  them  a  few  seconds  to  process  their  delivery  with  the 
administration  technician  and  then  they  leftthe  clinic. 

Patients  who  don't 
qualify  as  a  walk-in 
status  are  excluded  from 

the  model 

They  are  excluded  from  the  model  because  patients  who  do  not 
qualify  to  "walk-in"  are  told  to  make  a  future  appointment.  It  is 
assumed  that  there  infrequent  situations  like  this  and  that  these 
situations  do  not  take  much  of  the  administrative  technician's  time. 

Paperwork  completed 
priorto  vitals  checked 

The  patient  is  assumed  to  have  completed  their  paperwork  priorto 
having  the  nurse/technician  check  their  vitals. 

Prioritized  Queue 

It  is  assumed  that  patients  with  an  appointment  are  given  priority  to 
those  who  are  in  "walk-in"  status. 

No  Balking 

It  is  assumed  that  patients  entering  the  system  of  clinics  are 
committed  to  stay  in  the  clinic  upon  completion  and  not  leave  early. 

No  Transit  Time 

Forthe  purposes  of  this  study,  transit  time  is  not  included  in  the  model 
because  analyzing  wait-time  does  nottake  into  account  transit  time. 

Order  of  Availability 

The  model  assumes  that  when  one  process  node  is  unavailable,  the 
patient  moves  to  the  next  available  process  node.  Upon  completion  of 
that  process  node,  it  then  move  on  to  the  next  needed  process  node 
and  cycles  back  to  the  missed  process  node  when  it  becomes  available. 
Order:  Hearing  Conservation,  Check  Vitals,  Lab,  ECG,  X-ray,  2nd  Visit  to 
Nurse/Technician,  3rd  Visit  to  Nurse/Technician 

Stationary  Interrivals 

The  patient  interarrival  distribution  is  assumed  stationary  throughout 
the  day;  this  implies  no  peak  hours. 

Process  Limited  to 

Patient  Interactions 

It  is  assumed  that  process  times  are  all  accounted  with  patient 
interaction  only. 

No  Batching 

The  medical  staff  members  see  the  patient  one  at  a  time. 

Follow-Up  Appointment 

in  Person 

If  the  patient  needs  a  follow-up  appointment  after  seeing  the  doctor, 
then  it  is  assumed  that  the  patient  schedules  it  at  the  front  desk 
before  leaving  and  not  overthe  phone. 
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